11# Migrating from harp-python
22
33` harp-data ` is the successor to ` harp-python ` for reading Harp binary data files into
4- pandas DataFrames. The core concepts are unchanged — device schemas, register maps,
5- binary files — but the API has been reorganized to separate data reading from device
4+ pandas DataFrames. The core concepts of device schemas, register maps and binary files
5+ are unchanged, but the API has been reorganized to separate data reading from device
66communication.
77
88This guide covers the three workflows most users relied on in ` harp-python ` .
99
10- ---
11-
1210## Swap the package
1311
1412Replace the old dependency:
@@ -21,20 +19,18 @@ pip install harp-python
2119pip install harp-data
2220```
2321
24- If you want the full toolkit — serial transport, device client, and data reading — install
25- the umbrella package instead:
22+ For the full toolkit of serial transport, device client and data reading, install the
23+ umbrella package instead:
2624
2725``` sh
2826pip install harp
2927```
3028
31- ---
32-
3329## Loading a device schema at runtime
3430
3531In ` harp-python ` , ` harp.create_reader() ` accepted a dataset folder and handled
36- schema loading internally. ` create_dataset_reader ` is the direct replacement — it
37- finds the ` device.yml ` inside the folder automatically:
32+ schema loading internally. ` open_dataset ` is the direct replacement, and it finds the
33+ ` device.yml ` inside the folder automatically:
3834
3935``` python title="Before"
4036import harp
@@ -43,41 +39,44 @@ reader = harp.create_reader("session.harp")
4339```
4440
4541``` python title="After"
46- from harp.data import create_dataset_reader
42+ from harp.data import open_dataset
4743
48- reader = create_dataset_reader (" session.harp" )
44+ reader = open_dataset (" session.harp" )
4945```
5046
5147If the schema lives outside the data folder, pass it explicitly:
5248
5349``` python
54- reader = create_dataset_reader (" session.harp" , schema = " /path/to/device.yml" )
50+ reader = open_dataset (" session.harp" , schema = " /path/to/device.yml" )
5551```
5652
57- ### Accessing the device module
53+ ### Finding out what a session holds
5854
59- The reader holds a reference to the compiled device module at ` reader.device_module ` .
60- Use it to look up register classes by name — no need to keep a separate variable :
55+ ` reader.contents ` maps the name of every register with data in the folder to its address,
56+ in address order, which is the quickest way to see what was recorded :
6157
6258``` python
63- reader = create_dataset_reader(" session.harp" )
59+ reader = open_dataset(" session.harp" )
60+ print (reader.contents) # {'WhoAmI': 0, 'DigitalInputState': 32, ...}
61+ ```
6462
65- # access any register class through the reader
63+ The reader also holds the compiled device module at ` reader.device_module ` , so a register
64+ class can be reached without keeping a separate variable:
65+
66+ ``` python
6667df = reader.read(reader.device_module.AnalogData)
6768```
6869
6970!!! note
70- The old ` harp.read_schema() ` had no direct equivalent you needed to call separately.
71- ` create_dataset_reader ` handles schema loading in one step, matching the convenience
71+ The old ` harp.read_schema() ` had no direct equivalent that needed calling separately.
72+ ` open_dataset ` handles schema loading in one step, matching the convenience
7273 of the original API.
7374
74- ---
75-
7675## Reading a single register
7776
78- The old API gave you attribute access on the reader — ` reader.AnalogData.read() ` . The
79- new API inverts this: you call ` reader.read() ` and pass the register class or its
80- address as the argument.
77+ The old API allowed reads through attribute access on the reader. The new
78+ API inverts this, so ` reader.read() ` takes the register class, its name or
79+ its address as the argument.
8180
8281``` python title="Before"
8382# by attribute name
@@ -94,13 +93,22 @@ df = reader.registers[44].read()
9493# by register class (accessed through the reader)
9594df = reader.read(reader.device_module.AnalogData)
9695
96+ # by name, the direct analogue of the old string lookup
97+ df = reader.read(" AnalogData" )
98+
9799# by address
98100df = reader.read(44 )
99101```
100102
103+ Names resolve against the device address space rather than the module namespace, so a
104+ common register such as ` reader.read("WhoAmI") ` works even though a device module does
105+ not name it.
106+
101107### Absolute timestamps
102108
103- The ` epoch ` parameter moves from the reader constructor into the ` read() ` call:
109+ The ` epoch ` parameter keeps its name and moves from ` create_reader ` to ` open_dataset ` .
110+ ` harp-python ` also allowed it per read. ` harp-data ` sets it once for the dataset, so
111+ every register is read on the same clock.
104112
105113``` python title="Before"
106114reader = harp.create_reader(" session.harp" , epoch = harp.REFERENCE_EPOCH )
@@ -110,35 +118,53 @@ df = reader.AnalogData.read()
110118``` python title="After"
111119from harp.data import REFERENCE_EPOCH
112120
113- df = reader.read(reader.device_module.AnalogData, epoch = REFERENCE_EPOCH )
121+ reader = open_dataset(" session.harp" , epoch = REFERENCE_EPOCH )
122+ df = reader.read(reader.device_module.AnalogData)
114123```
115124
116125### Reading the whole session at once
117126
118- ` read_all() ` returns a dictionary of DataFrames keyed by register name. Registers
119- with no corresponding ` .bin ` file are skipped :
127+ There is no ` read_all() ` . Whole-session loading is a comprehension over ` contents ` ,
128+ which keeps the choice of what to load with the caller :
120129
121130``` python
122- everything: dict[ str , pd.DataFrame] = reader.read_all()
131+ everything = {name: reader.read(name) for name in reader.contents}
123132```
124133
134+ ### Bitmask registers lose their per-flag columns by default
135+
136+ This is the change most likely to break working code. ` harp-python ` always expanded a
137+ bitmask register into one boolean column per flag, so a script could select a flag by
138+ name. ` harp-data ` returns a single integer column instead, and expands the flags only
139+ when asked:
140+
141+ ``` python title="Before"
142+ led = reader.DigitalOutputSet.read()[" GP15" ]
143+ ```
144+
145+ ``` python title="After"
146+ led = reader.read(" DigitalOutputSet" , demux_bit_masks = True )[" GP15" ]
147+ ```
148+
149+ Group masks need no such flag. ` harp-python ` mapped each value to its member name, and
150+ ` harp-data ` decodes them by default, as a ` pd.Categorical ` rather than plain strings, so
151+ a comparison against a string still reads naturally.
152+
125153### Parameter reference
126154
127155| harp-python | harp-data | Notes |
128156| ---| ---| ---|
129- | ` keep_type=True ` | ` message_type=True ` | Renamed |
130- | ` epoch=REFERENCE_EPOCH ` | ` epoch=REFERENCE_EPOCH ` | Same |
131- | ` epoch=None ` | ` epoch=None ` (default) | Float seconds; same |
132- | — | ` decode_enums=True ` | New: enum fields as ` pd.Categorical ` |
133- | — | ` demux_bit_masks=False ` | New: expand bitmask flags into one column per flag |
134-
135- ---
157+ | ` keep_type=True ` | ` keep_type=True ` | Same, but the column is named ` message_type ` rather than ` MessageType ` |
158+ | ` epoch=REFERENCE_EPOCH ` | ` epoch=REFERENCE_EPOCH ` | Same, but set on ` open_dataset ` rather than per read |
159+ | ` epoch=None ` | ` epoch=None ` (default) | Float seconds, same |
160+ | inferred from the first frame | ` time_index=False ` | Needed for data carrying no timestamp |
161+ | always on | ` demux_bit_masks=True ` | Needed to keep per-flag columns |
162+ | always on | ` decode_enums=True ` (default) | Group mask values, now ` pd.Categorical ` |
136163
137164## Schemaless read
138165
139- If you have a raw ` .bin ` file and no schema — or you just want to inspect the data
140- quickly — the ` read() ` function works the same as before. Only the import path and
141- one parameter name change:
166+ For a raw ` .bin ` file with no schema, or a quick look at the data, the ` read() `
167+ function works the same as before. Only the import path changes:
142168
143169``` python title="Before"
144170import harp
@@ -151,38 +177,34 @@ df = harp.read("Behavior_44.bin", keep_type=True)
151177from harp.data import read
152178
153179df = read(" Behavior_44.bin" )
154- df = read(" Behavior_44.bin" , message_type = True )
180+ df = read(" Behavior_44.bin" , keep_type = True )
155181```
156182
157- Both functions infer the payload type and element count automatically from the first
158- frame — no register metadata needed.
159-
160- ---
183+ Both functions infer the payload type and element count from the frame, so no register
184+ metadata is needed. The new one assumes timestamped data, which is what a device sends,
185+ and takes ` time_index=False ` for the rare buffer that is not. It also takes ` epoch ` per
186+ call, since a single file has no dataset to set one on.
161187
162188## Going further: static device packages
163189
164- Loading a YAML at runtime is convenient, but for production workflows — or when you
165- want IDE autocompletion and type safety — Harp device packages are pre-compiled Python
166- modules that give you the same interface without any schema parsing at startup.
167-
168- A static device package installs its register map as a proper Python module. You
169- import it, pass it directly to ` DatasetReader ` , and the rest of the API is identical:
190+ Loading a YAML at runtime is convenient, but for production workflows, or where IDE
191+ autocompletion and type checking matter, a generated device package gives the same
192+ interface without parsing a schema at startup.
170193
171- ``` python
172- pip install harp- device- behavior
173- ```
194+ Such a package is an ordinary Python module under the ` harp.device ` namespace. Import
195+ it, pass it to ` open_dataset ` , and the rest of the API is identical:
174196
175197``` python
176- from harp.device.behavior import device as behavior
177- from harp.data import DatasetReader
198+ from harp.device import behavior
199+ from harp.data import open_dataset
178200
179- reader = DatasetReader(behavior, " session.harp" )
201+ reader = open_dataset( " session.harp" , behavior )
180202
181- # everything works the same
203+ # a register class now resolves statically
182204df = reader.read(behavior.AnalogData)
183- everything = reader.read_all()
184205```
185206
186- The static module is faster to start up and ships with stubs for autocompletion. See
207+ A generated module starts up faster and resolves under a type checker, which a module
208+ built from a schema at runtime cannot. See
187209[ Generating Registers from a Schema] ( ../examples/create_device_module/create_device_module.md )
188210for how device modules are structured.
0 commit comments