add option to fits reader to open L1 products also in UTC >L1 UTC per default (via keyword in header)#447
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #447 +/- ##
==========================================
- Coverage 72.23% 67.04% -5.19%
==========================================
Files 78 76 -2
Lines 8085 8136 +51
==========================================
- Hits 5840 5455 -385
- Misses 2245 2681 +436 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
samaloney
left a comment
There was a problem hiding this comment.
I'm not sure allowing operations in different time systems is a good idea. A product should either have time in OBT or UTC and the conversion should only be done once in the from_level0 or similar methods?
All products (>L0) will always have obt_timerange from products will have an utc_timerange but only if the time sys in header is UTC and not sure supporting on the fly conversion is a good idea because the spice kernel dependence.
This aren't blocker just thoughts
|
i agree that it is not ideal to provide some of the on the fly conversions. The auto converting most of the time (and if proper used never) will not happen as our process steps will do this explicit. In general this L1 product could have state depended time systems and i think it would be bad experience if the properties will raise an exception if used |
9b9c9dc to
748ae6a
Compare
748ae6a to
5a7c670
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds an opt-in reader behavior to interpret FITS time columns using the TIMESYS/DATE-OBS header context (via get_timeformat_from_TIMESYS=True) so L1 products can be opened in UTC when needed, while keeping SCET/OBT as the default for processing workflows.
Changes:
- Add
get_timeformat_from_TIMESYSoption toProduct(path)reading logic to choose UTC vs OBT-based offsets when reconstructingtime/timedel. - Update time-range properties and product combination logic to handle non-SCETime internal time formats.
- Rename
exposuretomin_exposureacross FITS header generation and tests.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| stixcore/util/scripts/end2end_testing.py | Adds local debug code in __main__ (currently breaks script execution). |
| stixcore/products/tests/test_factory.py | Adds test coverage for reading time formats (SCET vs UTC). |
| stixcore/products/product.py | Implements time-format selection from headers; updates timerange logic; renames exposure API; introduces UTC tz-aware constant. |
| stixcore/products/level3/flarelistproduct.py | Adds warning log when SCET is approximated from UTC using SPICE. |
| stixcore/products/level3/flarelist.py | Adds warning logs; renames exposure to min_exposure. |
| stixcore/processing/tests/test_publish.py | Updates tests to use min_exposure. |
| stixcore/processing/tests/test_end2end.py | Alters pytest markers for the end-to-end identical test. |
| stixcore/io/product_processors/tests/test_processors.py | Updates tests to use min_exposure and expect XPOSURE from it. |
| stixcore/io/product_processors/fits/processors.py | Writes XPOSURE from min_exposure; minor refactor of scet_timerange usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
samaloney
left a comment
There was a problem hiding this comment.
Seems ok, one test marker commented out?
| # @pytest.mark.end2end | ||
| @pytest.mark.remote_data | ||
| @pytest.mark.end2end |
add keyword: get_timeformat_from_TIMESYS to Product(path) method.
If set time and timedel column in data table are converted to timesystem provided in the TIMESYS header keyword.
some products properties like utc_timerange, min_exposure ... might do autoconversions based on the internal timesystem but it should be avoided to open L1 files in UTC for processing them further.
Use get_timeformat_from_TIMESYS=True only for reading excess to L1 data, read/write/combing might lead to timing accuracy issues