-
Notifications
You must be signed in to change notification settings - Fork 500
[SystemDS-#3524] Multi-threading of transformdecode/[SystemDS-#3521] Improved Feature Transformations #2275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…e done, test passes for Bin
…e done, test passes for Bin
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2275 +/- ##
============================================
- Coverage 72.95% 72.85% -0.10%
- Complexity 46062 46159 +97
============================================
Files 1479 1486 +7
Lines 172575 173144 +569
Branches 33776 33898 +122
============================================
+ Hits 125898 126149 +251
- Misses 37191 37481 +290
- Partials 9486 9514 +28 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Thank you for the patch. I will take a look into this next week @Isso-W. |
small error in test array
# Conflicts: # src/test/java/org/apache/sysds/test/functions/transform/ColumnDecoderMixedMethodsTest.java
This pull request introduces a new framework for column decoding in Apache SystemDS, with the addition of a base class
ColumnDecoder
and several specialized implementations (ColumnDecoderBin
,ColumnDecoderComposite
,ColumnDecoderRecode
,ColumnDecoderPassthrough
andColumnDecoderDummycode
). These changes provide a flexible and extensible structure for decoding encoded data in matrix-to-frame transformations. Below are the most important changes grouped by theme:Core Framework for Column Decoding
ColumnDecoder
as an abstract base class to define the structure for decoding operations, including methods for decoding (columnDecode
), handling sub-range decoding (subRangeDecoder
), and metadata initialization (initMetaData
). It also implementsExternalizable
for efficient serialization.Current Issues
ColumnDecoderDummycode
ist not supported yet, as well as the test caseColumnDecoderMixedMethodsTest
DecoderDummyCode
, it do not work together withDecoderRecode