hworker alternatives and similar packages
Based on the "Unclassified" category.
Alternatively, view hworker alternatives based on common mentions on social networks and blogs.
gotta-go-fast9.6 0.0 hworker VS gotta-go-fastA command line utility for practicing typing and measuring your WPM and accuracy.
elm-repl9.6 0.0 hworker VS elm-repla REPL for Elm
hascard9.5 2.7 hworker VS hascardflashcard TUI with markdown cards
pointfree9.3 4.3 hworker VS pointfreeMaintenance of the pointfree Hackage package.
orgstat9.0 3.2 hworker VS orgstatStatistics visualizer for org-mode
heroku-persistent8.7 0.0 hworker VS heroku-persistentParse DATABASE_URL into configuration types for Persistent
logging-effect8.7 0.0 hworker VS logging-effectA very general logging effect for Haskell
type-sets8.6 0.0 hworker VS type-setstype level sets
type-errors8.6 0.0 hworker VS type-errors:warning: tools for writing better type errors
load-balancing8.4 1.9 hworker VS load-balancingClient-side load balancing utilities.
safe8.3 0.0 hworker VS safeHaskell library for safe (pattern match free) functions
canteven-http8.3 1.7 hworker VS canteven-httpUtilities for HTTP programming.
ascii-art-to-unicode8.2 0.0 hworker VS ascii-art-to-unicodeSmall program to convert ASCII box art to Unicode box drawings.
rollbar-cli8.2 0.0 hworker VS rollbar-cliA group of libraries written in Haskell to communicate with Rollbar API.
bit-stream8.2 0.0 hworker VS bit-streamLazy infinite compact streams with cache-friendly O(1) indexing and applications for memoization
base-unicode-symbols8.0 1.2 hworker VS base-unicode-symbolsUnicode alternatives for common functions and operators
hackertyper7.9 0.0 hworker VS hackertyper"Hack" like a programmer in movies and games! Inspired by hackertyper.net
aeson-serialize7.7 0.0 hworker VS aeson-serializeFunctions for serializing a type that is an instance of ToJSON
domain7.6 0.0 hworker VS domainFocused domain model declaration toolkit for Haskell
SDL2-ttf7.5 0.0 hworker VS SDL2-ttfBinding to libSDL-ttf
ble7.3 0.0 hworker VS bleBluetooth Low Energy (BLE) peripherals
dependent-sum-templateTemplate Haskell code to generate instances of classes in dependent-sum package
rebase7.2 0.0 hworker VS rebaseA more progressive alternative to the "base" package
ispositive7.1 0.0 hworker VS ispositiveHaskell Module: Integer.IsPositive
rss7.1 0.0 hworker VS rssA library for generating RSS 2.0 feeds.
lrucaching7.0 1.0 hworker VS lrucachingHaskell implementation of an LRU cache
argon26.9 0.0 L3 hworker VS argon2Haskell bindings to libargon2 - the reference implementation of the Argon2 password-hashing function
dzen-dhall6.9 0.0 hworker VS dzen-dhallConfigure dzen2 bars in Dhall language
servant-streaming6.8 0.0 hworker VS servant-streamingSupport for servant requests and responses via the 'streaming' library
Deadpan-DDP6.7 0.0 hworker VS Deadpan-DDPA Haskell DDP Client
containers-unicode-symbolsUnicode alternatives for common functions and operators
google-oauth26.5 0.0 hworker VS google-oauth2Google OAuth2 token negotiation
MissingPy6.5 0.0 hworker VS MissingPySupport for calling Python from Haskell
gravatar6.5 0.0 hworker VS gravatarCompute gravatar urls for email addresses
preql6.5 0.0 hworker VS preqlexperiments with SQL & Haskell
semver-range6.4 0.0 hworker VS semver-rangeImplementation of semver and NPM-style semantic version ranges in Haskell
postgresql-simple-sop6.4 0.0 hworker VS postgresql-simple-sopGeneric functions for postgresql-simple
Ordinary6.4 0.0 hworker VS OrdinaryShort description of your package
reflex-dom-pandoc6.4 0.0 hworker VS reflex-dom-pandocRender Pandoc documents in reflex-dom
deque6.4 0.0 hworker VS dequeDouble-ended queues
dag6.3 0.0 hworker VS dagA well-typed Directed Acyclic Graph in Haskell
rerebase6.2 0.0 hworker VS rerebaseReexports from "base" with a bunch of other standard libraries
bitcoind-regtest6.1 0.0 hworker VS bitcoind-regtestA cilent for the bitcoind JSON-RPC interface
streaming-events6.1 0.0 hworker VS streaming-eventsClient-side consumption of a ServerEvent.
hasql-dynamic-statementsDynamic statements for Hasql
google-drive6.0 0.0 hworker VS google-driveGoogle Drive API access
network-carbon5.8 0.0 hworker VS network-carbonA Haskell implementation of the Carbon protocol (part of the Graphite monitoring tools)
Clean code begins in your IDE with SonarLint
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.
Do you think we are missing an alternative of hworker or a related project?
hworker is a Redis-backed persistent at-least-once queue library. It
is vaguely inspired by
sidekiq for Ruby. It is intended to be a
simple reliable mechanism for processing background tasks. The jobs
can be created by a Haskell application or any application that can
push JSON data structures of the right shape into a Redis queue. The
application that processes the jobs need not be the same one as the
application that creates them (they just need to be able to talk to
the same Redis server, and use the same serialization to/from JSON).
This has been running in one application sending email (using
hworker-ses) for several months. This is relatively low traffic
(transactional messages) most of the time, with spikes of 10k-30k
messages (mailing blasts).
The expiration of jobs is really important. It defaults to 120 seconds, which may be short depending on your application (for things like sending emails, it may be fine). The reason why this timeout is important is that if a job ever runs longer than this, the monitor will think that the job failed in some inexplicable way (like the server running the job died) and will add the job back to the queue to be run. Based on the semantics of this job processor, jobs running multiple times is not a failure case, but it's obviously not something you want to happen, so be sure to set the timeout to something reasonable for your application.
To define jobs, you define a serialized representation of the job, and
a function that runs the job, which returns a status. The behavior of
uncaught exceptions is defined when you create the worker - it can be
Retry. Jobs that return
Failure are removed
from the queue, whereas jobs that return
Retry are added again. The
only difference between a
Success and a
Failure is that a
Failure returns a message that is logged (ie, neither run again).
example directory in the repository.
This behavior of this queue processor is at-least-once.
We rely on the defined behavior of Redis for reliability. Once a job
queued, it is guaranteed to be run eventually, provided
some worker and monitor threads exist. If the worker thread that was
running a given job dies, the job will eventually be retried (if you
do not want this behavior, do not start any monitor threads). Once the
job completes, provided nothing kills the worker thread in the
intervening time, jobs that returned
Success will not be run again,
jobs that return
Failure will have their messages logged and will
not be run again, and jobs that return
Retry will be queued
again. If something kills the worker thread before these
acknowledgements go through, the job will be retried. Exceptions
triggered within the job cannot affect the worker thread - what they
do to the job is defined at startup (they can cause either a
Any deviations from this behavior are considered bugs that will be fixed.
Under the hood, we will have the following data structures in redis
name is set when you create the
hworker-jobs-name: list of json serialized job descriptions
hworker-progress-name: a hash of jobs that are in progress, mapping to time started
hworker-broken-name: a hash of jobs to time that couldn't be deserialized; most likely means you changed the serialization format with jobs still in queue, or you pointed different applications at the same queues.
hworker-failed-queue: a record of the jobs that failed (limited in size based on config).
In the following pseudo-code, I'm using
EXEC to indicate
atomic blocks of code. These are actually implemented with lua and
EVAL, but I think it's easier to read this way. If you want to see
what's actually happening, just read the code - it's not very long!
When a worker wants to do work, the following happens:
now = TIME MULTI v = RPOP hworker-jobs-name if v HSET hworker-progress-name v now EXEC v
When it completes the job, it does the following:
v = JOB HDEL hwork-progress v
If the job returned
Retry, the following occurs:
v = JOB t = START_TIME MULTI LPUSH hwork-jobs v HDEL hwork-progress t EXEC
A monitor runs on another thread that will re-run jobs that stay in progress for too long (as that indicates that something unknown went wrong). The operation that it runs periodically is:
keys = HKEYS (or HSCAN) hwork-progress for keys as v: started = HGET hwork-progress v if started < TIME - timeout MULTI RPUSH hwork-jobs v HDEL hwork-progress v EXEC
Note that what the monitor does and
Retry is slightly different -
the monitor puts jobs on the front of the queue, whereas
them on the back.
Primary Libraries Used
- Daniel Patterson (@dbp - [email protected])