conduit-network-stream alternatives and similar packages
Based on the "Conduit" category.
Alternatively, view conduit-network-stream alternatives based on common mentions on social networks and blogs.
-
pool-conduit
Persistence interface for Haskell allowing multiple storage methods. -
http-conduit
An HTTP client engine, intended as a base layer for more user-friendly packages. -
conduit-combinators
Type classes for mapping, folding, and traversing monomorphic containers -
twitter-conduit
Twitter API package for Haskell, including enumerator interfaces and Streaming API supports. -
html-conduit
Various XML utility packages for Haskell -
csv-conduit
Flexible, fast and constant-space CSV library for Haskell using conduits -
hreq-conduit
A type dependent highlevel HTTP client library inspired by servant-client. -
simple-conduit
A simple streaming I/O library based on monadic folds -
stomp-conduit
Message-oriented Middleware for Haskell -
binary-conduit
binary serialization interface for conduit -
conduit-audio
Use conduit to process/manipulate/convert audio -
crypto-conduit
Conduit interface for cryptographic operations (from crypto-api). -
http-conduit-browser
Browser interface to the http-conduit package -
fsnotify-conduit
Get filesystem notifications as a stream of events -
imagesize-conduit
Conduit sink to efficiently determine image dimensions -
rss-conduit
Streaming parser/renderer for the RSS 2.0 standard. -
cryptonite-conduit
conduit bridge for cryptonite -
udp-conduit
[DONE] Simple fire-and-forget style conduit parts (sources/sinks) for UDP traffic -
cryptohash-conduit
Simple conduit function for cryptohash -
hw-conduit-merges
Additional merge / join combinators for Conduit -
conduit-concurrent-map
Concurrent, order-preserving mapping Conduit for Haskell -
jsonrpc-conduit
Library for building JSON-RPC 2.0 servers. -
conduit-zstd
Conduit interface to zstandard compression -
conduit-tokenize-attoparsec
Conduits for tokenizing streams. -
ftp-conduit
FTP file commands using the conduit interface -
ascii85-conduit
Conduit for encoding ByteString into Ascii85 -
potoki-conduit
Integration of "potoki" and "conduit"
Updating dependencies is time-consuming.
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.
Do you think we are missing an alternative of conduit-network-stream or a related project?
README
Conduits
The Conduit package by Michael Snoyman is an approach to the streaming data problem. It is meant as an alternative to enumerators/iterators, hoping to address the same issues with different trade-offs based on real-world experience with enumerators.
Conduit statements always have a Source
and a Sink
for data. These can be
anything read- or writeable, for example a file, a network connection or simply
stdin
/stdout
. Between the Source
and the Sink
you can use conduits to
modify stream elements.
A very simple example for an application using conduits would be the "uppercase
copy" of a text file: The sourceFile
streams its characters to the uppercase
conduit before all elements are stored to a new file by the sinkFile
command.
sourceFile "myfile.txt" $= uppercase $$ sinkFile "mycopy.txt"
For more detailed information about conduits, head over to Michaels github and take a look at the haddock documentation on hackage.
A Network Protocol with Conduits
The main goal of this library is a simple-to-use network library based on conduits which supports sending and receiving of single and multiple "messages" through one single server/client connection.
Part of the Conduit project is the network-conduit package, which provides basic functions for streaming binary data over a network connection:
type Application m = AppData m -> m ()
appSource :: AppData m -> Source m ByteString
appSink :: AppData m -> Sink ByteString m ()
runTCPServer :: (MonadIO m, MonadBaseControl IO m)
=> ServerSettings m -> Application m -> m ()
runTCPClient :: (MonadIO m, MonadBaseControl IO m)
=> ClientSettings m -> Application m -> m ()
For example, you could copy a file over network by using sourceFile
and
appSink
in the client...
runTCPClient (clientSettings ..) $ \appData ->
sourceFile "myfile.txt" $$ appSink appData
... and appSource
with sinkFile
in the server:
runTCPServer (serverSettings ..) $ \appData ->
appSource appData $$ sinkFile "myfile-networkcopy.txt"
With this very rudimentary API, it is difficult to distinguish between separate
ByteString
packages ("messages"). To solve this problem, the
conduit-network-stream
library wraps every ByteString
into a tiny header
which contains its exact length. This header is designed such that it's not only
possible to send single ByteString
packages but also lists of ByteString
s:
send1 :: (Monad m, Sendable a m)
=> AppData m -> Source (Stream m) a -> m ()
sendList :: (Monad m, Sendable a m)
=> AppData m -> Source (Stream m) a -> m ()
Using these functions, it is very simple to receive those ByteString
packages.
All you have to supply is a ByteString
sink (Note that these are lazy
ByteString
s):
receive :: AppData m
-> Sink ByteString (Stream m) b
-> m (ResumableSource (Stream m) ByteString, b)
To avoid two different functions for "fresh" (AppData m
) and "old"
(ResumableSource
) sources I introduced a type class with instances for both:
class Streamable source m where
receive :: source
-> Sink ByteString (Stream m) b
-> m (ResumableSource (Stream m) ByteString, b)
instance Streamable (AppData m) m where ...
instance Streamable (ResumableSource (Stream m) ByteString) m where ...
With this function you could then for example define a function which would receive protocol-buffer messages:
receiveProtoBuff :: (Streamable source m, ReflectDescriptor msg, Wire msg)
=> source
-> Sink msg (Stream m) b
-> m (ResumableSource (Stream m) ByteString, b)
receiveProtoBuff src sink = receive src $ toMsg =$ sink
where
toMsg = Data.Conduits.List.mapMaybe $ \bs ->
case messageGet bs of
Right (msg,_) -> Just msg
Left _ -> Nothing
A simple client/server application with this package (and -XOverloadedStrings
)
could look like this:
client = runTCPClient myClientSettings $ \appData -> do
-- send one single `ByteString`
send1 appData $
Data.Conduit.yield "Hello world!"
-- send a `ByteString` list
sendList appData $
mapM_ Data.Conduit.yield ["This", "is", "a", "list", "of", "words."]
server = runTCPServer myServerSettings $ \appData -> do
(next, bs) <- receive appData $
Data.Conduit.List.consume
-- print: "Hello world!"
liftIO $
mapM_ print bs
(next', bs') <- receive next $
Data.Conduit.List.consume
-- print: "This"
-- "is"
-- "a"
-- "list"
-- "of"
-- "words."
liftIO $
mapM_ print bs'
-- close the conduit stream
close next'
Note that receive
automatically terminates as soon as the end of a list/block
is reached, without closing the connection. All conduits inside a receive
will
only manipulate the elements of this one block/list.
Also note that I deviated from the original Source
/Sink
model by accepting a
Sink
/Source
argument to the send
/recive
functions instead of using the
standard $$
operator to pipe one to the other. This has a couple of reasons:
Convenience: I found that in practice, there is very little difference between a
Source
and aSink
conduit. Usually it doesn't matter whether you writesrc $= conduit $$ sink
orsrc $$ conduit =$ sink
. Having just one argument to each function simplifies the API a bit.Type safety: Using the
send
andreceive
functions I can wrap the "inner" conduit into aStream
newtype. This makes it impossible to use a stream with this encoding outside of its intended context. (Despite this point I still exported the~~
operator, a lifted version of$$
. There are a number of scenarios where this is useful, but try to avoid whenever possible)Immutable source: Applying a conduit to a
Source
changes the source for all "resumed" instances aswell. This would make it impossible to decode the stream properly later on. By usingreceive
this problem can be avoided.
The Protocol
A detailed explanation of the used protocol/header is at the top of
Conduit.Network.Stream.Header
.