Basic manipulation of pylipd.LiPD objects#
Preamble#
Goals:#
Extract a LiPD time series for analysis
Remove/pop LiPD datasets from an existing
LiPDobject
Reading Time: 5 minutes
Keywords#
LiPD; query
Pre-requisites#
None. This tutorial assumes basic knowledge of Python and Pandas. If you are not familiar with this coding language and the Pandas library, check out this tutorial: http://linked.earth/ec_workshops_py/.
Relevant Packages#
Pandas, pylipd
Data Description#
This notebook uses the following datasets, in LiPD format:
Nurhati, I. S., Cobb, K. M., & Di Lorenzo, E. (2011). Decadal-scale SST and salinity variations in the central tropical Pacific: Signatures of natural and anthropogenic climate change. Journal of Climate, 24(13), 3294–3308. doi:10.1175/2011jcli3852.1
PAGES2k Consortium (2017): A global multiproxy database for temperature reconstructions of the Common Era. Sci Data 4, 170088. doi:10.1038/sdata.2017.88
from pylipd.lipd import LiPD
Demonstration#
Extract time series data from LiPD formatted datasets#
If you are famliar with the R utilities, one useful functions is the ability to expand “timeseries” structures. This capability was also present in the previous iteration of the Python utilities and PyLiPD retains this compatbility to ease the transition.
If you’re unsure about what a “timeseries” is in the LiPD context, read this page.
Working with one dataset#
First, let’s load a single dataset:
data_path = '../data/Ocn-Palmyra.Nurhati.2011.lpd'
D = LiPD()
D.load(data_path)
Loading 1 LiPD files
0%| | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 28.93it/s]
Loaded..
Now let’s get all the timeseries for this dataset. Note that the get_timeseries function requires to pass the dataset names. This is useful if you only want to expand only one dataset from your LiPD object. You can also use the function get_all_dataset_names in the call to expand all datasets:
ts_list = D.get_timeseries(D.get_all_dataset_names())
type(ts_list)
Extracting timeseries from dataset: Ocn-Palmyra.Nurhati.2011 ...
dict
Note that the above function returns a dictionary that organizes the extracted timeseries by dataset name:
ts_list.keys()
dict_keys(['Ocn-Palmyra.Nurhati.2011'])
Each timeseries is then stored into a list of dictionaries that preserve essential metadata for each time/depth and value pair:
type(ts_list['Ocn-Palmyra.Nurhati.2011'])
list
Although the information is present, it is not easy to navigate or query across the various list. One simple way of doing so is to return the list into a Pandas.DataFrame:
ts_list, df = D.get_timeseries(D.get_all_dataset_names(), to_dataframe=True)
df
Extracting timeseries from dataset: Ocn-Palmyra.Nurhati.2011 ...
| mode | time_id | archiveType | pub1_author | pub1_url | pub1_urldate | pub1_institution | pub1_citeKey | pub1_title | pub2_author | ... | paleoData_sensorSpecies | paleoData_useInGlobalTemperatureAnalysis | paleoData_sensorGenus | paleoData_proxy | paleoData_pages2kID | paleoData_inCompilation | paleoData_iso2kUI | paleoData_qCCertification | paleoData_interpretation | paleoData_ocean2kID | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | paleoData | age | Coral | I.S. Nurhati | [https://www.ncdc.noaa.gov/paleo/study/8609] | 2010.0 | World Data Center for Paleoclimatology | nurhati2010httpswwwncdcnoaagovpaleostudy8609Da... | World Data Center for Paleoclimatology | Kim M. Cobb;Intan S. Nurhati;Emanuele Di Lorenzo | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 1 | paleoData | age | Coral | I.S. Nurhati | [https://www.ncdc.noaa.gov/paleo/study/8609] | 2010.0 | World Data Center for Paleoclimatology | nurhati2010httpswwwncdcnoaagovpaleostudy8609Da... | World Data Center for Paleoclimatology | Kim M. Cobb;Intan S. Nurhati;Emanuele Di Lorenzo | ... | lutea | False | Porites | d18O | NaN | NaN | NaN | NaN | NaN | NaN |
| 2 | paleoData | age | Coral | I.S. Nurhati | [https://www.ncdc.noaa.gov/paleo/study/8609] | 2010.0 | World Data Center for Paleoclimatology | nurhati2010httpswwwncdcnoaagovpaleostudy8609Da... | World Data Center for Paleoclimatology | Kim M. Cobb;Intan S. Nurhati;Emanuele Di Lorenzo | ... | lutea | True | Porites | Sr/Ca | Ocn_129 | Ocean2k_v1.0.0 | CO11NUPM01BT1 | MNE, NJA | [{'scope': 'climate', 'variableDetail': 'sea@s... | PacificNurhati2011 |
| 3 | paleoData | age | Coral | I.S. Nurhati | [https://www.ncdc.noaa.gov/paleo/study/8609] | 2010.0 | World Data Center for Paleoclimatology | nurhati2010httpswwwncdcnoaagovpaleostudy8609Da... | World Data Center for Paleoclimatology | Kim M. Cobb;Intan S. Nurhati;Emanuele Di Lorenzo | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 4 | paleoData | age | Coral | I.S. Nurhati | [https://www.ncdc.noaa.gov/paleo/study/8609] | 2010.0 | World Data Center for Paleoclimatology | nurhati2010httpswwwncdcnoaagovpaleostudy8609Da... | World Data Center for Paleoclimatology | Kim M. Cobb;Intan S. Nurhati;Emanuele Di Lorenzo | ... | lutea | False | Porites | d18O | NaN | NaN | CO11NUPM01B | MNE, NJA | [{'direction': 'negative', 'scope': 'climate',... | NaN |
5 rows × 88 columns
You can now use all the pandas functionalities for filtering and querying dataframes. First, let’s have a look at the available properties, which corresponds to the column headers:
df.columns
Index(['mode', 'time_id', 'archiveType', 'pub1_author', 'pub1_url',
'pub1_urldate', 'pub1_institution', 'pub1_citeKey', 'pub1_title',
'pub2_author', 'pub2_pages', 'pub2_dataUrl', 'pub2_publisher',
'pub2_year', 'pub2_journal', 'pub2_volume', 'pub2_DOI', 'pub2_title',
'pub2_citeKey', 'pub3_author', 'pub3_issue', 'pub3_title', 'pub3_DOI',
'pub3_citeKey', 'pub3_publisher', 'pub3_year', 'pub3_pages',
'pub3_dataUrl', 'pub3_journal', 'pub3_volume', 'googleDataURL',
'lipdVersion', 'originalDataURL', 'dataContributor',
'googleSpreadSheetKey', 'createdBy', 'studyName', 'hasUrl',
'geo_meanLon', 'geo_meanLat', 'geo_meanElev', 'geo_type', 'geo_ocean',
'geo_siteName', 'geo_pages2kRegion', 'dataSetName',
'googleMetadataWorksheet', 'tableType', 'paleoData_paleoDataTableName',
'paleoData_measurementTableMD5', 'paleoData_measurementTableName',
'paleoData_googleWorkSheetKey', 'paleoData_filename', 'year',
'yearUnits', 'paleoData_description', 'paleoData_variableName',
'paleoData_wDSPaleoUrl', 'paleoData_hasArchiveType_label',
'paleoData_hasMeanValue', 'paleoData_hasMinValue',
'paleoData_resolution_hasMaxValue', 'paleoData_resolution_hasMinValue',
'paleoData_resolution_hasMeanValue',
'paleoData_resolution_hasMedianValue', 'paleoData_resolution_units',
'paleoData_hasMedianValue', 'paleoData_dataType', 'paleoData_TSid',
'paleoData_variableType', 'paleoData_inferredVariableType',
'paleoData_number', 'paleoData_hasMaxValue', 'paleoData_missingValue',
'paleoData_units', 'paleoData_values', 'paleoData_proxyObservationType',
'paleoData_notes', 'paleoData_sensorSpecies',
'paleoData_useInGlobalTemperatureAnalysis', 'paleoData_sensorGenus',
'paleoData_proxy', 'paleoData_pages2kID', 'paleoData_inCompilation',
'paleoData_iso2kUI', 'paleoData_qCCertification',
'paleoData_interpretation', 'paleoData_ocean2kID'],
dtype='object')
Let’s have a look at the paleoData_variableName column to see what’s available:
df['paleoData_variableName']
0 year
1 d18O
2 Sr/Ca
3 year
4 d18O
Name: paleoData_variableName, dtype: object
All columns get extracted, hence why year is extracted as a paleo variable, with its associated values stored in paleoData_values. Notice that there is also two variables names d18O. Since this is a coral record, it stands to reason that one corresponds to the measured \(\delta^{18}O\) of the coral and the other the \(\delta^{18}O\) of the seawater. Let’s have a look at the notes field:
df[['paleoData_variableName','paleoData_notes']]
| paleoData_variableName | paleoData_notes | |
|---|---|---|
| 0 | year | NaN |
| 1 | d18O | d18Osw (residuals calculated from coupled SrCa... |
| 2 | Sr/Ca | ; paleoData_variableName changed - was origina... |
| 3 | year | NaN |
| 4 | d18O | Duplicate of modern d18O record presented in C... |
In fact, one is for the measurement on the coral and the other for seawater. Querying on this small dataset is not necessary; however, it can become useful when looking at a collection of files as shown in the next example (working with multiple datasets).
To extract by row index (here extracting for Sr_Ca):
df_cut = df.iloc[4,:]
df_cut
mode paleoData
time_id age
archiveType Coral
pub1_author I.S. Nurhati
pub1_url [https://www.ncdc.noaa.gov/paleo/study/8609]
...
paleoData_inCompilation NaN
paleoData_iso2kUI CO11NUPM01B
paleoData_qCCertification MNE, NJA
paleoData_interpretation [{'direction': 'negative', 'scope': 'climate',...
paleoData_ocean2kID NaN
Name: 4, Length: 88, dtype: object
df_cut['paleoData_variableName']
'd18O'
This can be very useful when working with the Pyleoclim software since a Pyleoclim.Series can be initialized from the information contained in df_cut. Working with PyLiPD and Pyleoclim is the subject of several tutorials.
Working with such a large dataframe can be overwhelming and not needed in some cases. Therefore, PyLiPD has a nifty function called get_timeseries_essentials that grabs information about the dataset, its geographical location, the time/depth values, the variable information, including archive and proxy:
df_essential = D.get_timeseries_essentials()
df_essential
| dataSetName | archiveType | geo_meanLat | geo_meanLon | geo_meanElev | paleoData_variableName | paleoData_values | paleoData_units | paleoData_proxy | paleoData_proxyGeneral | time_variableName | time_values | time_units | depth_variableName | depth_values | depth_units | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Ocn-Palmyra.Nurhati.2011 | Coral | 5.87 | -162.13 | -10.0 | d18O | [0.39, 0.35, 0.35, 0.35, 0.36, 0.22, 0.33, 0.3... | permil | d18O | None | year | [1998.21, 1998.13, 1998.04, 1997.96, 1997.88, ... | yr AD | None | None | None |
| 1 | Ocn-Palmyra.Nurhati.2011 | Coral | 5.87 | -162.13 | -10.0 | d18O | [-5.41, -5.47, -5.49, -5.43, -5.48, -5.53, -5.... | permil | d18O | None | year | [1998.29, 1998.21, 1998.13, 1998.04, 1997.96, ... | yr AD | None | None | None |
| 2 | Ocn-Palmyra.Nurhati.2011 | Coral | 5.87 | -162.13 | -10.0 | Sr_Ca | [8.96, 8.9, 8.91, 8.94, 8.92, 8.89, 8.87, 8.81... | mmol/mol | Sr/Ca | None | year | [1998.29, 1998.21, 1998.13, 1998.04, 1997.96, ... | yr AD | None | None | None |
The metadata (i.e., the column names) available through this function will always remain the same and are as follows:
df_essential.columns
Index(['dataSetName', 'archiveType', 'geo_meanLat', 'geo_meanLon',
'geo_meanElev', 'paleoData_variableName', 'paleoData_values',
'paleoData_units', 'paleoData_proxy', 'paleoData_proxyGeneral',
'time_variableName', 'time_values', 'time_units', 'depth_variableName',
'depth_values', 'depth_units'],
dtype='object')
Working with multiple datasets#
path = '../data/Pages2k/'
D_dir = LiPD()
D_dir.load_from_dir(path)
Loading 16 LiPD files
0%| | 0/16 [00:00<?, ?it/s]
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:16 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
31%|███▏ | 5/16 [00:00<00:00, 42.90it/s]
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:49 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
62%|██████▎ | 10/16 [00:00<00:00, 35.70it/s]
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:47:20 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
88%|████████▊ | 14/16 [00:00<00:00, 36.57it/s]
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:12 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
100%|██████████| 16/16 [00:00<00:00, 37.45it/s]
Loaded..
Let’s expand into our essential dataframe:
df_dir = D_dir.get_timeseries_essentials()
Let’s have a look at the dataframe:
df_dir.head()
| dataSetName | archiveType | geo_meanLat | geo_meanLon | geo_meanElev | paleoData_variableName | paleoData_values | paleoData_units | paleoData_proxy | paleoData_proxyGeneral | time_variableName | time_values | time_units | depth_variableName | depth_values | depth_units | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Ocn-RedSea.Felis.2000 | Coral | 27.8500 | 34.3200 | -6.0 | d18O | [-4.12, -3.82, -3.05, -3.02, -3.62, -3.96, -3.... | permil | d18O | None | year | [1995.583, 1995.417, 1995.25, 1995.083, 1994.9... | yr AD | None | None | None |
| 1 | Eur-CoastofPortugal.Abrantes.2011 | Marine sediment | 41.1000 | -8.9000 | -80.0 | temperature | [15.235, 15.329, 15.264, 15.376, 15.4, 15.129,... | degC | alkenone | None | year | [971.19, 982.672, 991.858, 1001.044, 1010.23, ... | yr AD | None | None | None |
| 2 | Eur-LakeSilvaplana.Trachsel.2010 | Lake sediment | 46.5000 | 9.8000 | 1791.0 | temperature | [0.181707222, 0.111082797, 0.001382129, -0.008... | degC | reflectance | None | year | [1175, 1176, 1177, 1178, 1179, 1180, 1181, 118... | yr AD | None | None | None |
| 3 | Arc-Kongressvatnet.D'Andrea.2012 | Lake sediment | 78.0217 | 13.9311 | 94.0 | temperature | [5.9, 5.1, 6.1, 5.3, 4.3, 4.8, 3.8, 4.8, 4.3, ... | degC | alkenone | None | year | [2008, 2004, 2000, 1996, 1990, 1987, 1982, 197... | yr AD | None | None | None |
| 4 | Arc-Kongressvatnet.D'Andrea.2012 | Lake sediment | 78.0217 | 13.9311 | 94.0 | Uk37 | [-0.65, -0.67, -0.65, -0.67, -0.69, -0.68, -0.... | None | None | None | year | [2008, 2004, 2000, 1996, 1990, 1987, 1982, 197... | yr AD | None | None | None |
The size of this dataframe is:
df_dir.shape
(25, 16)
So we expanded into 25 timeseries.
Let’s have a look at the available variables:
df_dir['paleoData_variableName'].unique()
array(['d18O', 'temperature', 'Uk37', 'MXD', 'Mg_Ca', 'notes', 'trsgi',
'uncertainty_temperature'], dtype=object)
Let’s assume we are only interested in the temperature data:
df_temp = df_dir[df_dir['paleoData_variableName']=='temperature']
df_temp.head()
| dataSetName | archiveType | geo_meanLat | geo_meanLon | geo_meanElev | paleoData_variableName | paleoData_values | paleoData_units | paleoData_proxy | paleoData_proxyGeneral | time_variableName | time_values | time_units | depth_variableName | depth_values | depth_units | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Eur-CoastofPortugal.Abrantes.2011 | Marine sediment | 41.1000 | -8.9000 | -80.0 | temperature | [15.235, 15.329, 15.264, 15.376, 15.4, 15.129,... | degC | alkenone | None | year | [971.19, 982.672, 991.858, 1001.044, 1010.23, ... | yr AD | None | None | None |
| 2 | Eur-LakeSilvaplana.Trachsel.2010 | Lake sediment | 46.5000 | 9.8000 | 1791.0 | temperature | [0.181707222, 0.111082797, 0.001382129, -0.008... | degC | reflectance | None | year | [1175, 1176, 1177, 1178, 1179, 1180, 1181, 118... | yr AD | None | None | None |
| 3 | Arc-Kongressvatnet.D'Andrea.2012 | Lake sediment | 78.0217 | 13.9311 | 94.0 | temperature | [5.9, 5.1, 6.1, 5.3, 4.3, 4.8, 3.8, 4.8, 4.3, ... | degC | alkenone | None | year | [2008, 2004, 2000, 1996, 1990, 1987, 1982, 197... | yr AD | None | None | None |
| 5 | Ocn-AlboranSea436B.Nieto-Moreno.2013 | Marine sediment | 36.2053 | -4.3133 | -1108.0 | temperature | [18.79, 19.38, 19.61, 18.88, 18.74, 19.25, 18.... | degC | alkenone | None | year | [1999.07, 1993.12, 1987.17, 1975.26, 1963.36, ... | yr AD | None | None | None |
| 7 | Eur-Stockholm.Leijonhufvud.2009 | Documents | 59.3200 | 18.0600 | 10.0 | temperature | [-1.7212, -1.6382, -0.6422, 0.1048, -0.7252, -... | degC | historical | None | year | [1502, 1503, 1504, 1505, 1506, 1507, 1508, 150... | yr AD | None | None | None |
df_temp.shape
(11, 16)
which leaves us with 11 timeseries.
Let’s assume that you want everything that is not related to time, depth, and uncertainty. To keep the rows that are relevant to our problem, you can use the DataFrame.query function available in Pandas:
df_filt = df_dir.query("paleoData_variableName in ('temperature','MXD','Mg_Ca','d18O','trsgi', 'Uk37')")
df_filt.head()
| dataSetName | archiveType | geo_meanLat | geo_meanLon | geo_meanElev | paleoData_variableName | paleoData_values | paleoData_units | paleoData_proxy | paleoData_proxyGeneral | time_variableName | time_values | time_units | depth_variableName | depth_values | depth_units | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Ocn-RedSea.Felis.2000 | Coral | 27.8500 | 34.3200 | -6.0 | d18O | [-4.12, -3.82, -3.05, -3.02, -3.62, -3.96, -3.... | permil | d18O | None | year | [1995.583, 1995.417, 1995.25, 1995.083, 1994.9... | yr AD | None | None | None |
| 1 | Eur-CoastofPortugal.Abrantes.2011 | Marine sediment | 41.1000 | -8.9000 | -80.0 | temperature | [15.235, 15.329, 15.264, 15.376, 15.4, 15.129,... | degC | alkenone | None | year | [971.19, 982.672, 991.858, 1001.044, 1010.23, ... | yr AD | None | None | None |
| 2 | Eur-LakeSilvaplana.Trachsel.2010 | Lake sediment | 46.5000 | 9.8000 | 1791.0 | temperature | [0.181707222, 0.111082797, 0.001382129, -0.008... | degC | reflectance | None | year | [1175, 1176, 1177, 1178, 1179, 1180, 1181, 118... | yr AD | None | None | None |
| 3 | Arc-Kongressvatnet.D'Andrea.2012 | Lake sediment | 78.0217 | 13.9311 | 94.0 | temperature | [5.9, 5.1, 6.1, 5.3, 4.3, 4.8, 3.8, 4.8, 4.3, ... | degC | alkenone | None | year | [2008, 2004, 2000, 1996, 1990, 1987, 1982, 197... | yr AD | None | None | None |
| 4 | Arc-Kongressvatnet.D'Andrea.2012 | Lake sediment | 78.0217 | 13.9311 | 94.0 | Uk37 | [-0.65, -0.67, -0.65, -0.67, -0.69, -0.68, -0.... | None | None | None | year | [2008, 2004, 2000, 1996, 1990, 1987, 1982, 197... | yr AD | None | None | None |
df_filt.shape
(22, 16)
Which leaves us with 22 timeseries.
Removing and popping datasets out of a LiPD object#
You can also remove (i.e., delete the corresponding dataset from the LiPD object) or pop (i.e., delete the corresponding dataset from the LiPD object and return the dataset) datasets from a LiPD object. Note that these functionalities behave similarly as the functions with the same names on Python lists. These functions underpin more adavanced filtering and querying capabilities that we will discuss in later tutorials.
First let’s make a copy of D_dir:
D_test = D_dir.copy()
print(D_test.get_all_dataset_names())
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:16 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:49 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:47:20 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:46:12 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
Failed to convert Literal lexical form to value. Datatype=http://www.w3.org/2001/XMLSchema#date, Converter=<function parse_xsd_date at 0x7ff69dcbd630>
Traceback (most recent call last):
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/term.py", line 2163, in _castLexicalToPython
return conv_func(lexical) # type: ignore[arg-type]
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/rdflib/xsd_datetime.py", line 593, in parse_xsd_date
return parse_date(date_string if not minus else ("-" + date_string))
File "/usr/share/miniconda/envs/pylipd/lib/python3.10/site-packages/isodate/isodates.py", line 193, in parse_date
raise ISO8601Error("Unrecognised ISO 8601 date format: %r" % datestring)
isodate.isoerror.ISO8601Error: Unrecognised ISO 8601 date format: '2022-08-23 23:41:56 U'
['Ocn-RedSea.Felis.2000', 'Eur-CoastofPortugal.Abrantes.2011', 'Eur-LakeSilvaplana.Trachsel.2010', 'Arc-Kongressvatnet.D_Andrea.2012', 'Ocn-AlboranSea436B.Nieto-Moreno.2013', 'Ocn-SinaiPeninsula_RedSea.Moustafa.2000', 'Eur-Stockholm.Leijonhufvud.2009', 'Eur-NorthernScandinavia.Esper.2012', 'Eur-SpannagelCave.Mangini.2005', 'Eur-NorthernSpain.Martin-Chivelet.2011', 'Asi-SourthAndMiddleUrals.Demezhko.2007', 'Ocn-PedradeLume-CapeVerdeIslands.Moses.2006', 'Ocn-FeniDrift.Richter.2009', 'Eur-SpanishPyrenees.Dorado-Linan.2012', 'Ant-WAIS-Divide.Severinghaus.2012', 'Eur-FinnishLakelands.Helama.2014']
And let’s remove Eur-Stockholm.Leijonhufvud.2009, which corresponds to the last entry in the list above:
D_test.remove('Eur-Stockholm.Leijonhufvud.2009')
print(D_test.get_all_dataset_names())
['Ocn-RedSea.Felis.2000', 'Eur-CoastofPortugal.Abrantes.2011', 'Eur-LakeSilvaplana.Trachsel.2010', 'Arc-Kongressvatnet.D_Andrea.2012', 'Ocn-AlboranSea436B.Nieto-Moreno.2013', 'Ocn-SinaiPeninsula_RedSea.Moustafa.2000', 'Eur-NorthernScandinavia.Esper.2012', 'Eur-SpannagelCave.Mangini.2005', 'Eur-NorthernSpain.Martin-Chivelet.2011', 'Asi-SourthAndMiddleUrals.Demezhko.2007', 'Ocn-PedradeLume-CapeVerdeIslands.Moses.2006', 'Ocn-FeniDrift.Richter.2009', 'Eur-SpanishPyrenees.Dorado-Linan.2012', 'Ant-WAIS-Divide.Severinghaus.2012', 'Eur-FinnishLakelands.Helama.2014']
Now let’s pop Eur-NorthernScandinavia.Esper.2012 from D_test:
d_eur = D_test.pop('Eur-NorthernScandinavia.Esper.2012')
Now let’s have a look at d_eur:
print(d_eur.get_all_dataset_names())
['Eur-NorthernScandinavia.Esper.2012']
It contains the dataset we are expecting. Let’s have a look at D_test:
print(D_test.get_all_dataset_names())
['Ocn-RedSea.Felis.2000', 'Eur-CoastofPortugal.Abrantes.2011', 'Eur-LakeSilvaplana.Trachsel.2010', 'Arc-Kongressvatnet.D_Andrea.2012', 'Ocn-AlboranSea436B.Nieto-Moreno.2013', 'Ocn-SinaiPeninsula_RedSea.Moustafa.2000', 'Eur-SpannagelCave.Mangini.2005', 'Eur-NorthernSpain.Martin-Chivelet.2011', 'Asi-SourthAndMiddleUrals.Demezhko.2007', 'Ocn-PedradeLume-CapeVerdeIslands.Moses.2006', 'Ocn-FeniDrift.Richter.2009', 'Eur-SpanishPyrenees.Dorado-Linan.2012', 'Ant-WAIS-Divide.Severinghaus.2012', 'Eur-FinnishLakelands.Helama.2014']
The dataset was removed from `D_test` in the process. Hence, it's always prudent to make a copy of the original object when using the `remove` and `pop` functionalities.
If can also remove/pop more than one dataset at a time:
rem = ['Ocn-RedSea.Felis.2000','Ant-WAIS-Divide.Severinghaus.2012']
D_test.remove(rem)
print(D_test.get_all_dataset_names())
['Eur-CoastofPortugal.Abrantes.2011', 'Eur-LakeSilvaplana.Trachsel.2010', 'Arc-Kongressvatnet.D_Andrea.2012', 'Ocn-AlboranSea436B.Nieto-Moreno.2013', 'Ocn-SinaiPeninsula_RedSea.Moustafa.2000', 'Eur-SpannagelCave.Mangini.2005', 'Eur-NorthernSpain.Martin-Chivelet.2011', 'Asi-SourthAndMiddleUrals.Demezhko.2007', 'Ocn-PedradeLume-CapeVerdeIslands.Moses.2006', 'Ocn-FeniDrift.Richter.2009', 'Eur-SpanishPyrenees.Dorado-Linan.2012', 'Eur-FinnishLakelands.Helama.2014']