tstoolbox.tstoolbox.convert_index

tstoolbox.tstoolbox.convert_index(to, interval=None, epoch='julian', input_ts='-', columns=None, start_date=None, end_date=None, round_index=None, dropna='no', clean=False, names=None, source_units=None, target_units=None, skiprows=None)

Convert datetime to/from Julian dates from different epochs.

Parameters
  • to (str) – One of ‘number’ or ‘datetime’. If ‘number’, the source time-series should have a datetime index to convert to a number. If ‘datetime’, source data should be a number and the converted index will be datetime.

  • interval

    [optional, defaults to None]

    The interval parameter defines the unit time. One of the pandas offset codes. The default of ‘None’ will set the unit time for all defined epochs to daily except ‘unix’ which will defaults to seconds.

    You can give any smaller unit time than daily for all defined epochs except ‘unix’ which requires an interval less than seconds. For an epoch that begins with an arbitrary date, you can use any interval equal to or smaller than the frequency of the time-series.

    Alias

    Description

    B

    business day

    C

    custom business day (experimental)

    D

    calendar day

    W

    weekly

    M

    month end

    BM

    business month end

    CBM

    custom business month end

    MS

    month start

    BMS

    business month start

    CBMS

    custom business month start

    Q

    quarter end

    BQ

    business quarter end

    QS

    quarter start

    BQS

    business quarter start

    A

    year end

    BA

    business year end

    AS

    year start

    BAS

    business year start

    H

    hourly

    T

    minutely

    S

    secondly

    L

    milliseconds

    U

    microseconds

    N

    nanoseconds

    Weekly has the following anchored frequencies:

    Alias

    Description

    W-SUN

    weekly frequency (sundays). Same as ‘W’.

    W-MON

    weekly frequency (mondays)

    W-TUE

    weekly frequency (tuesdays)

    W-WED

    weekly frequency (wednesdays)

    W-THU

    weekly frequency (thursdays)

    W-FRI

    weekly frequency (fridays)

    W-SAT

    weekly frequency (saturdays)

    Quarterly frequencies (Q, BQ, QS, BQS) and annual frequencies (A, BA, AS, BAS) have the following anchoring suffixes:

    Alias

    Description

    -DEC

    year ends in December (same as ‘Q’ and ‘A’)

    -JAN

    year ends in January

    -FEB

    year ends in February

    -MAR

    year ends in March

    -APR

    year ends in April

    -MAY

    year ends in May

    -JUN

    year ends in June

    -JUL

    year ends in July

    -AUG

    year ends in August

    -SEP

    year ends in September

    -OCT

    year ends in October

    -NOV

    year ends in November

  • epoch (str) –

    [optional, defaults to ‘julian’]

    Can be one of, ‘julian’, ‘reduced’, ‘modified’, ‘truncated’, ‘dublin’, ‘cnes’, ‘ccsds’, ‘lop’, ‘lilian’, ‘rata_die’, ‘mars_sol_date’, ‘unix’, or a date and time.

    If supplying a date and time, most formats are recognized, however the closer the format is to ISO 8601 the better. Also should check and make sure date was parsed as expected. If supplying only a date, the epoch starts at midnight the morning of that date.

    The ‘unix’ epoch uses a default interval of seconds, and all other defined epochs use a default interval of ‘daily’.

    epoch

    Epoch

    Calculation

    Notes

    julian

    4713-01-01:12 BCE

    JD

    reduced

    1858-11-16:12

    JD - 2400000

    [ 1 ] [ 2 ]

    modified

    1858-11-17:00

    JD - 2400000.5

    SAO 1957

    truncated

    1968-05-24:00

    floor (JD - 2440000.5)

    NASA 1979, integer

    dublin

    1899-12-31:12

    JD - 2415020

    IAU 1955

    cnes

    1950-01-01:00

    JD - 2433282.5

    CNES [ 3 ]

    ccsds

    1958-01-01:00

    JD - 2436204.5

    CCSDS [ 3 ]

    lop

    1992-01-01:00

    JD - 2448622.5

    LOP [ 3 ]

    lilian

    1582-10-15[13]

    floor (JD - 2299159.5)

    Count of days of the Gregorian calendar, integer

    rata_die

    0001-01-01[13] proleptic Gregorian calendar

    floor (JD - 1721424.5)

    Count of days of the Common Era, integer

    mars_sol

    1873-12-29:12

    (JD - 2405522) /1.02749

    Count of Martian days

    unix

    1970-01-01 T00:00:00

    JD - 2440587.5

    seconds

    1

    . Hopkins, Jeffrey L. (2013). Using Commercial Amateur Astronomical Spectrographs, p. 257, Springer Science & Business Media, ISBN 9783319014425

    2

    . Palle, Pere L., Esteban, Cesar. (2014). Asteroseismology, p. 185, Cambridge University Press, ISBN 9781107470620

    3(1,2,3)

    . Theveny, Pierre-Michel. (10 September 2001). “Date Format” The TPtime Handbook. Media Lab.

  • input_ts (str) –

    [optional, required if using Python API, default is ‘-‘ (stdin)]

    Whether from a file or standard input, data requires a header of column names. The default header is the first line of the input, but this can be changed using the ‘skiprows’ option.

    Most separators will be automatically detected. Most common date formats can be used, but the closer to ISO 8601 date/time standard the better.

    Command line:

    +-------------------------+------------------------+
    | --input_ts=filename.csv | to read 'filename.csv' |
    +-------------------------+------------------------+
    | --input_ts='-'          | to read from standard  |
    |                         | input (stdin).         |
    +-------------------------+------------------------+
    
    In many cases it is better to use redirection rather that use
    `--input_ts=filename.csv`.  The following are identical:
    
    From a file:
    
        command subcmd --input_ts=filename.csv
    
    From standard input:
    
        command subcmd --input_ts=- < filename.csv
    
    The BEST way since you don't have to include `--input_ts=-` because
    that is the default:
    
        command subcmd < filename.csv
    
    Can also combine commands by piping:
    
        command subcmd < filename.csv | command subcmd1 > fileout.csv
    

    As Python Library:

    You MUST use the `input_ts=...` option where `input_ts` can be one
    of a [pandas DataFrame, pandas Series, dict, tuple,
    list, StringIO, or file name].
    
    If result is a time series, returns a pandas DataFrame.
    

  • columns

    [optional, defaults to all columns, input filter]

    Columns to select out of input. Can use column names from the first line header or column numbers. If using numbers, column number 1 is the first data column. To pick multiple columns; separate by commas with no spaces. As used in tstoolbox pick command.

    This solves a big problem so that you don’t have to create a data set with a certain order, you can rearrange columns when data is read in.

  • start_date (str) –

    [optional, defaults to first date in time-series, input filter]

    The start_date of the series in ISOdatetime format, or ‘None’ for beginning.

  • end_date (str) –

    [optional, defaults to last date in time-series, input filter]

    The end_date of the series in ISOdatetime format, or ‘None’ for end.

  • round_index

    [optional, default is None which will do nothing to the index, output format]

    Round the index to the nearest time point. Can significantly improve the performance since can cut down on memory and processing requirements, however be cautious about rounding to a very course interval from a small one. This could lead to duplicate values in the index.

  • dropna (str) –

    [optional, defauls it ‘no’, input filter]

    Set dropna to ‘any’ to have records dropped that have NA value in any column, or ‘all’ to have records dropped that have NA in all columns. Set to ‘no’ to not drop any records. The default is ‘no’.

  • clean

    [optional, default is False, input filter]

    The ‘clean’ command will repair an index, removing duplicate index values and sorting.

  • skiprows (list-like or integer or callable) –

    [optional, default is None which will infer header from first line, input filter]

    Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file.

    If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be

    lambda x: x in [0, 2].

  • names

    [optional, default is None, input filter]

    If None, the column names are taken from the first row after ‘skiprows’ from the input dataset.

  • source_units

    [optional, default is None, transformation]

    If unit is specified for the column as the second field of a ‘:’ delimited column name, then the specified units and the ‘source_units’ must match exactly.

    Any unit string compatible with the ‘pint’ library can be used.

  • target_units

    [optional, default is None, transformation]

    The main purpose of this option is to convert units from those specified in the header line of the input into ‘target_units’.

    The units of the input time-series or values are specified as the second field of a ‘:’ delimited name in the header line of the input or in the ‘source_units’ keyword.

    Any unit string compatible with the ‘pint’ library can be used.

    This option will also add the ‘target_units’ string to the column names.

  • tablefmt (str) –

    [optional, default is ‘csv’, output format]

    The table format. Can be one of ‘csv’, ‘tsv’, ‘plain’, ‘simple’, ‘grid’, ‘pipe’, ‘orgtbl’, ‘rst’, ‘mediawiki’, ‘latex’, ‘latex_raw’ and ‘latex_booktabs’.