reedsolomon alternatives and similar packages
Based on the "Data" category.
Alternatively, view reedsolomon alternatives based on common mentions on social networks and blogs.
-
lens
Lenses, Folds, and Traversals - Join us on web.libera.chat #haskell-lens -
semantic-source
Parsing, analyzing, and comparing source code across many languages -
text
Haskell library for space- and time-efficient operations over Unicode text. -
code-builder
Packages for defining APIs, running them, generating client code and documentation. -
unordered-containers
Efficient hashing-based container types -
compendium-client
Mu (μ) is a purely functional framework for building micro services. -
cassava
A CSV parsing and encoding library optimized for ease of use and high performance -
holmes
A reference library for constraint-solving with propagators and CDCL. -
primitive
This package provides various primitive memory-related operations. -
binary
Efficient, pure binary serialisation using ByteStrings in Haskell. -
resource-pool
A high-performance striped resource pooling implementation for Haskell -
discrimination
Fast linear time sorting and discrimination for a large class of data types -
audiovisual
Extensible records, variants, structs, effects, tangles -
dependent-sum
Dependent sums and supporting typeclasses for comparing and displaying them -
IORefCAS
A collection of different packages for CAS based data structures. -
dependent-map
Dependently-typed finite maps (partial dependent products) -
reflection
Reifies arbitrary Haskell terms into types that can be reflected back into terms -
orgmode-parse
Attoparsec parser combinators for parsing org-mode structured text! -
streaming
An optimized general monad transformer for streaming applications, with a simple prelude of functions -
safecopy
An extension to Data.Serialize with built-in version control -
text-icu
This package provides the Haskell Data.Text.ICU library, for performing complex manipulation of Unicode text. -
scientific
Arbitrary-precision floating-point numbers represented using scientific notation -
uuid-types
A Haskell library for creating, printing and parsing UUIDs
TestGPT | Generating meaningful tests for busy devs
Do you think we are missing an alternative of reedsolomon or a related project?
README
Reed-Solomon
Reed-Solomon Erasure Coding in Haskell, with speeds exceeding multiple GB/s/cpu core implemented in pure Haskell (and some SIMD C/assembler).
This is a Haskell port of the GolangReedSolomon library released by Klaus Post, wich is a port of the JavaReedSolomon library released by Backblaze, with some additional optimizations.
For an introduction on erasure coding, see the post on the Backblaze blog.
Package home: https://github.com/NicolasT/reedsolomon
Performance
Performance depends mainly on the number of parity shards. In rough terms, doubling the number of parity shards will double the encoding time.
Here are the throughput numbers with some different selections of data and parity shards. For reference each shard is 1MB random data, and 1 CPU core is used for encoding.
Data Parity Parity SSSE3 MB/s AVX2 MB/s 5240%3641,663987,24 10220%3951,014444,44 10440%1821,161927,90 502040%398,09431,78
Example of performance on Intel(R) Core(TM) i7-4600U CPU @ 3.30GHz - 2 physical cores, 4 logical cores (note: /proc/cpuinfo
mentions 2.10GHz only). The example uses 10 blocks with 16MB data each and 4 parity blocks.
Threads SSSE3 MB/s AVX2 MB/s Speed 11551,891588,88100%
Links
- Backblaze Open Sources Reed-Solomon Erasure Coding Source Code.
- GolangReedSolomon. Compatible Go library by Klaus Post.
- JavaReedSolomon. Compatible java library by Backblaze.
- go-erasure. A similar library using cgo, slower in my tests.
- Screaming Fast Galois Field Arithmetic. Basis for SSE3 optimizations.
License
This code, as the original GolangReedSolomon and JavaReedSolomon is published under an MIT license. See LICENSE file for more information.
*Note that all licence references and agreements mentioned in the reedsolomon README section above
are relevant to that project's source code only.