solution for : still struggling to load properly

[ltroussellier]

playing with https://github.com/LinkedEarth/PaleoBooks/blob/master/notebooks/EDC_demo.ipynb

sep="/s+"

seems to load the txt correctly

[CommonClimate]

Merci Laurent! I will play with that tomorrow as part of other upgrades to the notebook, which is now out of step with the latest Pyleoclim releases (0.8.x).

[CommonClimate]

Hi Laurent,
actually, with that separator I still encounter problems:

(top is your method loading the txt file, bottom is the csv I manually extracted)

What does your output look like? Perhaps it's a pandas version thing too. In this environment I am running 1.4.1

[ltroussellier]

Hi,
i have done it like this :

main differences i can spot :
1/ I used the "txt" file, and not the "csv" one.
2/ the encoding, but i think it's link to the file format (in csv, panda can actually tell)
3/ the starting row

But in fact, the top panda dataframe you showed is maybe good : there is a lot of NaN in the data before the age 38.37379

After dropping Nan:

Hope this helps toward a simpler notebook

[CommonClimate]

Hello @ltroussellier . I tried and failed your solution again with my version of pandas. Can you tell me which one you are using? For now, the notebook (updated to work with v0.8.1), still uses the hand-made csv file.

[ltroussellier]

In fact, I use a personnel kernel on a remote jupyter to test some librairies so, not the clearest pip list but here the complete list :
Pandas : 1.2.4

Package                       Version
----------------------------- --------------------------
adal                          1.2.7
adlfs                         0.7.5
affine                        2.3.0
aiobotocore                   1.2.2
aiohttp                       3.7.4
aioitertools                  0.7.1
alembic                       1.6.2
amqp                          5.0.6
anyio                         3.1.0
appdirs                       1.4.4
argon2-cffi                   20.1.0
asciitree                     0.3.3
asn1crypto                    1.4.0
astropy                       4.2.1
async-generator               1.10
async-timeout                 3.0.1
attrs                         21.2.0
av                            8.0.3
awscli                        1.19.77
azure-core                    1.14.0
azure-datalake-store          0.0.51
azure-identity                1.6.0
azure-storage-blob            12.8.1
Babel                         2.9.1
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
batchspawner                  1.1.0
bcrypt                        3.2.0
beautifulsoup4                4.9.3
billiard                      3.6.4.0
bleach                        3.3.0
blinker                       1.4
blosc                         1.10.2
bokeh                         2.3.2
boto3                         1.17.77
botocore                      1.20.77
Bottleneck                    1.3.2
branca                        0.4.2
brotlipy                      0.7.0
cached-property               1.5.2
cachetools                    4.2.2
cachey                        0.2.1
Cartopy                       0.19.0.post1
celery                        5.0.5
certifi                       2021.10.8
certipy                       0.1.3
cf-xarray                     0.5.2
cffi                          1.14.5
cfgrib                        0.9.9.0
cftime                        1.5.0
cgen                          2020.1
chardet                       4.0.0
ciso                          0.1.0
click                         7.1.2
click-didyoumean              0.0.3
click-plugins                 1.1.1
click-repl                    0.1.6
cligj                         0.7.1
cloudpickle                   1.6.0
cmip6-preprocessing           0.0.0
cmocean                       2.0
colorama                      0.4.3
colorcet                      2.0.6
colorspacious                 1.1.2
configobj                     5.0.6
croniter                      0.3.36
cryptography                  3.4.7
cycler                        0.10.0
cytoolz                       0.11.0
dask                          2021.5.0
dask-gateway                  0.9.0
dask-glm                      0.2.0
dask-jobqueue                 0.7.2
dask-kubernetes               2021.3.1
dask-labextension             5.0.1
dask-ml                       1.9.0
datacube                      1.8.3
datashader                    0.12.1
datashape                     0.5.4
decorator                     5.0.9
defusedxml                    0.7.1
descartes                     1.1.0
distributed                   2021.5.0
docker                        5.0.0
docker-pycreds                0.4.0
docopt                        0.6.2
docrep                        0.2.7
docutils                      0.15.2
donfig                        0.6.0
eccodes                       1.2.0
entrypoints                   0.3
eofs                          1.4.0
erddapy                       1.0.0
ESMPy                         8.1.0
et-xmlfile                    1.1.0
fastapi                       0.65.1
fasteners                     0.16
fastjmd95                     0.2.1
fastprogress                  1.0.0
Fiona                         1.8.18
fsspec                        2021.5.0
future                        0.18.2
gcsfs                         2021.5.0
GDAL                          3.2.1
geocube                       0.0.16
geopandas                     0.9.0
geoviews                      0.0.0+g33876c88.gitarchive
gitdb                         4.0.7
GitPython                     3.1.17
google-auth                   1.30.0
google-auth-oauthlib          0.4.4
graphviz                      0.16
greenlet                      1.1.0
gsw                           3.4.0
h11                           0.12.0
h5netcdf                      0.11.0
h5py                          3.2.1
HeapDict                      1.0.1
holoviews                     1.14.3
httptools                     0.1.1
hvplot                        0.7.1
idna                          2.10
imagecodecs                   2021.3.31
imageio                       2.9.0
importlib-metadata            4.0.1
importlib-resources           5.1.3
intake                        0.6.2
intake-esm                    2021.1.15
intake-geopandas              0.2.4
intake-stac                   0.0.0
intake-xarray                 0.5.0
ipykernel                     5.5.5
ipyleaflet                    0.13.6
ipympl                        0.8.8
ipyspin                       0.1.5
ipython                       7.23.1
ipython-genutils              0.2.0
ipywidgets                    7.6.3
isodate                       0.6.0
jdcal                         1.4.1
jedi                          0.18.0
Jinja2                        3.0.1
jmespath                      0.10.0
joblib                        1.0.1
json5                         0.9.5
jsonschema                    3.2.0
jupyter-bokeh                 3.0.0
jupyter-client                6.1.12
jupyter-core                  4.7.1
jupyter-panel-proxy           0.1.0
jupyter-resource-usage        0.6.0
jupyter-server                1.7.0
jupyter-server-mathjax        0.2.2
jupyter-server-proxy          3.0.2
jupyter-telemetry             0.1.0
jupyterhub                    1.4.1
jupyterlab                    3.0.16
jupyterlab-execute-time       2.0.2
jupyterlab-git                0.30.1
jupyterlab-pygments           0.1.2
jupyterlab-server             2.5.2
jupyterlab-widgets            1.0.0
kiwisolver                    1.3.1
kombu                         5.0.2
kubernetes                    17.17.0
kubernetes-asyncio            12.1.0
lark-parser                   0.11.3
llvmlite                      0.36.0
locket                        0.2.0
lz4                           3.1.3
Mako                          1.1.4
Markdown                      3.3.4
MarkupSafe                    2.0.1
marshmallow                   3.12.1
marshmallow-oneofschema       2.1.0
matplotlib                    3.4.2
matplotlib-inline             0.1.2
mercantile                    1.2.1
MetPy                         1.0.1
mistune                       0.8.4
monotonic                     1.5
mpi4py                        3.0.3
msal                          1.12.0
msal-extensions               0.3.0
msgpack                       1.0.2
msrest                        0.6.21
multidict                     5.1.0
multipledispatch              0.6.0
munch                         2.5.0
mypy-extensions               0.4.3
natsort                       7.1.1
nb-conda-kernels              2.3.1
nbclassic                     0.2.8
nbclient                      0.5.3
nbconvert                     6.0.7
nbdime                        3.0.0
nbformat                      5.1.3
nbgitpuller                   0.9.0
nbstripout                    0.4.0
nc-time-axis                  1.2.0
nest-asyncio                  1.5.1
netCDF4                       1.5.6
networkx                      2.5
node                          0.9.26
nodejs                        0.1.1
notebook                      6.4.0
npm                           0.1.1
numba                         0.53.1
numcodecs                     0.7.3
numpy                         1.20.3
oauthlib                      3.0.1
odict                         1.7.0
olefile                       0.46
openpyxl                      3.0.7
optional-django               0.1.0
packaging                     20.9
pamela                        1.0.0
pandas                        1.2.4
pandocfilters                 1.4.2
panel                         0.11.3
param                         1.10.1
paramiko                      2.7.2
parcels                       2.2.2
parso                         0.8.2
partd                         1.2.0
patsy                         0.5.1
pendulum                      2.1.2
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        8.2.0
Pint                          0.17
pip                           20.3.4
plumber                       1.6
pooch                         1.3.0
portalocker                   1.7.0
prefect                       0.14.19
progressbar2                  3.53.1
prometheus-client             0.10.1
prompt-toolkit                3.0.18
psutil                        5.8.0
psycopg2                      2.8.6
ptyprocess                    0.7.0
pyarrow                       4.0.0
pyasn1                        0.4.8
pyasn1-modules                0.2.7
pycamhd                       0.7.0
pycparser                     2.20
pyct                          0.4.6
pycurl                        7.43.0.6
pydantic                      1.8.2
Pydap                         3.2.2
pyerfa                        1.7.3
Pygments                      2.9.0
pyhdf                         0.10.3
PyJWT                         2.1.0
pykdtree                      1.3.4
pymbolic                      2021.1
PyNaCl                        1.4.0
pyOpenSSL                     20.0.1
pyorbital                     1.6.1
pyparsing                     2.4.7
pyproj                        3.0.1
PyQt5                         5.12.3
PyQt5-sip                     4.19.18
PyQtChart                     5.12
PyQtWebEngine                 5.12.1
pyresample                    1.19.0
pyrsistent                    0.17.3
pyshp                         2.1.3
PySocks                       1.7.1
pyspectral                    0.10.5
pystac                        0.5.6
pystac-client                 0.1.1
python-box                    5.3.0
python-dateutil               2.7.5
python-dotenv                 0.17.1
python-editor                 1.0.4
python-geotiepoints           1.2.1
python-gist                   0.10.4
python-gnupg                  0.4.7
python-json-logger            2.0.1
python-slugify                5.0.2
python-utils                  2.5.6
pytools                       2021.2.6
pytz                          2021.1
pytzdata                      2020.1
pyviz-comms                   2.0.1
PyWavelets                    1.1.1
PyYAML                        5.4.1
pyzmq                         22.0.3
qgrid                         1.3.1
rasterio                      1.2.3
rechunker                     0.4.2
redis                         3.5.3
requests                      2.25.1
requests-oauthlib             1.3.0
retrying                      1.3.3
rio-cogeo                     2.0.1
rioxarray                     0.4.0
rsa                           4.7.2
Rtree                         0.9.7
ruamel.yaml                   0.17.4
ruamel.yaml.clib              0.2.2
s3fs                          2021.5.0
s3transfer                    0.4.2
sat-search                    0.3.0
sat-stac                      0.4.1
satpy                         0.28.1
scikit-image                  0.18.1
scikit-learn                  0.24.2
scipy                         1.6.3
seaborn                       0.11.2
Send2Trash                    1.5.0
setuptools                    49.6.0.post20210108
Shapely                       1.7.1
simpervisor                   0.4
six                           1.16.0
smmap                         3.0.5
sniffio                       1.2.0
snuggs                        1.4.7
sortedcontainers              2.4.0
soupsieve                     2.0.1
sparse                        0.12.0
spatialpandas                 0.3.6
SQLAlchemy                    1.4.15
sshtunnel                     0.3.1
starlette                     0.14.2
statsmodels                   0.12.2
strict-rfc3339                0.7
supermercado                  0.2.0
tabulate                      0.8.9
tblib                         1.7.0
terminado                     0.10.0
testpath                      0.5.0
text-unidecode                1.3
threadpoolctl                 2.1.0
tifffile                      2021.4.8
tiledb                        0.8.9
toml                          0.10.2
toolz                         0.11.1
tornado                       6.1
tqdm                          4.60.0
traitlets                     5.0.5
traittypes                    0.2.1
trollimage                    1.15.0
trollsift                     0.3.5
typing-extensions             3.7.4.3
Unidecode                     1.2.0
urllib3                       1.26.4
uvicorn                       0.13.4
uvloop                        0.15.2
vine                          5.0.0
voila                         0.2.10
watchgod                      0.6
wcwidth                       0.2.5
webencodings                  0.5.1
WebOb                         1.8.7
websocket-client              0.57.0
websockets                    8.1
wheel                         0.36.2
widgetsnbextension            3.5.1
wrapt                         1.12.1
xarray                        0.18.2
xarray-leaflet                0.1.13
xarray-spatial                0.2.2
xarrayutils                   0.1.3
xcape                         0+unknown
xcube                         0.8.1
xesmf                         0.5.3
xgcm                          0.5.1
xhistogram                    0.2.0
xlayers                       0.2.2
xmitgcm                       0.5.1
xpublish                      0.1.0
xrft                          0.3.0
yarl                          1.6.3
zarr                          2.8.3
zict                          2.0.0
zipp                          3.4.1
zope.component                5.0.1
zope.deprecation              4.4.0
zope.event                    4.5.0
zope.hookable                 5.1.0
zope.interface                5.4.0
zope.lifecycleevent           4.3