Portability | portable |
---|---|
Stability | experimental |
Maintainer | dons@galois.com |
Safe Haskell | Trustworthy |
- The
ByteString
type - Introducing and eliminating
ByteString
s - Basic interface
- Transforming ByteStrings
- Reducing
ByteString
s (folds) - Building ByteStrings
- Substrings
- Predicates
- Searching ByteStrings
- Indexing ByteStrings
- Zipping and unzipping ByteStrings
- Ordered ByteStrings
- Low level conversions
- I/O with
ByteString
s
A time and space-efficient implementation of lazy byte vectors
using lists of packed Word8
arrays, suitable for high performance
use, both in terms of large data quantities, or high speed
requirements. Byte vectors are encoded as lazy lists of strict Word8
arrays of bytes. They provide a means to manipulate large byte vectors
without requiring the entire vector be resident in memory.
Some operations, such as concat, append, reverse and cons, have better complexity than their Data.ByteString equivalents, due to optimisations resulting from the list spine structure. And for other operations lazy ByteStrings are usually within a few percent of strict ones, but with better heap usage. For data larger than the available memory, or if you have tight memory constraints, this module will be the only option. The default chunk size is 64k, which should be good in most circumstances. For people with large L2 caches, you may want to increase this to fit your cache.
This module is intended to be imported qualified
, to avoid name
clashes with Prelude functions. eg.
import qualified Data.ByteString.Lazy as B
Original GHC implementation by Bryan O'Sullivan.
Rewritten to use UArray
by Simon Marlow.
Rewritten to support slices and use ForeignPtr
by David Roundy.
Polished and extended by Don Stewart.
Lazy variant by Duncan Coutts and Don Stewart.
- data ByteString
- empty :: ByteString
- singleton :: Word8 -> ByteString
- pack :: [Word8] -> ByteString
- unpack :: ByteString -> [Word8]
- fromChunks :: [ByteString] -> ByteString
- toChunks :: ByteString -> [ByteString]
- cons :: Word8 -> ByteString -> ByteString
- cons' :: Word8 -> ByteString -> ByteString
- snoc :: ByteString -> Word8 -> ByteString
- append :: ByteString -> ByteString -> ByteString
- head :: ByteString -> Word8
- uncons :: ByteString -> Maybe (Word8, ByteString)
- last :: ByteString -> Word8
- tail :: ByteString -> ByteString
- init :: ByteString -> ByteString
- null :: ByteString -> Bool
- length :: ByteString -> Int64
- map :: (Word8 -> Word8) -> ByteString -> ByteString
- reverse :: ByteString -> ByteString
- intersperse :: Word8 -> ByteString -> ByteString
- intercalate :: ByteString -> [ByteString] -> ByteString
- transpose :: [ByteString] -> [ByteString]
- foldl :: (a -> Word8 -> a) -> a -> ByteString -> a
- foldl' :: (a -> Word8 -> a) -> a -> ByteString -> a
- foldl1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- foldl1' :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- foldr :: (Word8 -> a -> a) -> a -> ByteString -> a
- foldr1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- concat :: [ByteString] -> ByteString
- concatMap :: (Word8 -> ByteString) -> ByteString -> ByteString
- any :: (Word8 -> Bool) -> ByteString -> Bool
- all :: (Word8 -> Bool) -> ByteString -> Bool
- maximum :: ByteString -> Word8
- minimum :: ByteString -> Word8
- scanl :: (Word8 -> Word8 -> Word8) -> Word8 -> ByteString -> ByteString
- mapAccumL :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)
- mapAccumR :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)
- repeat :: Word8 -> ByteString
- replicate :: Int64 -> Word8 -> ByteString
- cycle :: ByteString -> ByteString
- iterate :: (Word8 -> Word8) -> Word8 -> ByteString
- unfoldr :: (a -> Maybe (Word8, a)) -> a -> ByteString
- take :: Int64 -> ByteString -> ByteString
- drop :: Int64 -> ByteString -> ByteString
- splitAt :: Int64 -> ByteString -> (ByteString, ByteString)
- takeWhile :: (Word8 -> Bool) -> ByteString -> ByteString
- dropWhile :: (Word8 -> Bool) -> ByteString -> ByteString
- span :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- break :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- group :: ByteString -> [ByteString]
- groupBy :: (Word8 -> Word8 -> Bool) -> ByteString -> [ByteString]
- inits :: ByteString -> [ByteString]
- tails :: ByteString -> [ByteString]
- split :: Word8 -> ByteString -> [ByteString]
- splitWith :: (Word8 -> Bool) -> ByteString -> [ByteString]
- isPrefixOf :: ByteString -> ByteString -> Bool
- isSuffixOf :: ByteString -> ByteString -> Bool
- elem :: Word8 -> ByteString -> Bool
- notElem :: Word8 -> ByteString -> Bool
- find :: (Word8 -> Bool) -> ByteString -> Maybe Word8
- filter :: (Word8 -> Bool) -> ByteString -> ByteString
- partition :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- index :: ByteString -> Int64 -> Word8
- elemIndex :: Word8 -> ByteString -> Maybe Int64
- elemIndices :: Word8 -> ByteString -> [Int64]
- findIndex :: (Word8 -> Bool) -> ByteString -> Maybe Int64
- findIndices :: (Word8 -> Bool) -> ByteString -> [Int64]
- count :: Word8 -> ByteString -> Int64
- zip :: ByteString -> ByteString -> [(Word8, Word8)]
- zipWith :: (Word8 -> Word8 -> a) -> ByteString -> ByteString -> [a]
- unzip :: [(Word8, Word8)] -> (ByteString, ByteString)
- copy :: ByteString -> ByteString
- getContents :: IO ByteString
- putStr :: ByteString -> IO ()
- putStrLn :: ByteString -> IO ()
- interact :: (ByteString -> ByteString) -> IO ()
- readFile :: FilePath -> IO ByteString
- writeFile :: FilePath -> ByteString -> IO ()
- appendFile :: FilePath -> ByteString -> IO ()
- hGetContents :: Handle -> IO ByteString
- hGet :: Handle -> Int -> IO ByteString
- hGetNonBlocking :: Handle -> Int -> IO ByteString
- hPut :: Handle -> ByteString -> IO ()
- hPutNonBlocking :: Handle -> ByteString -> IO ByteString
- hPutStr :: Handle -> ByteString -> IO ()
The ByteString
type
data ByteString Source
A space-efficient representation of a Word8 vector, supporting many
efficient operations. A ByteString
contains 8-bit characters only.
Instances of Eq, Ord, Read, Show, Data, Typeable
Introducing and eliminating ByteString
s
O(1) The empty ByteString
singleton :: Word8 -> ByteStringSource
O(1) Convert a Word8
into a ByteString
pack :: [Word8] -> ByteStringSource
O(n) Convert a '[Word8]' into a ByteString
.
unpack :: ByteString -> [Word8]Source
O(n) Converts a ByteString
to a '[Word8]'.
fromChunks :: [ByteString] -> ByteStringSource
O(c) Convert a list of strict ByteString
into a lazy ByteString
toChunks :: ByteString -> [ByteString]Source
O(n) Convert a lazy ByteString
into a list of strict ByteString
Basic interface
cons :: Word8 -> ByteString -> ByteStringSource
O(1) cons
is analogous to '(:)' for lists.
cons' :: Word8 -> ByteString -> ByteStringSource
O(1) Unlike cons
, 'cons\'' is
strict in the ByteString that we are consing onto. More precisely, it forces
the head and the first chunk. It does this because, for space efficiency, it
may coalesce the new byte onto the first 'chunk' rather than starting a
new 'chunk'.
So that means you can't use a lazy recursive contruction like this:
let xs = cons\' c xs in xs
You can however use cons
, as well as repeat
and cycle
, to build
infinite lazy ByteStrings.
snoc :: ByteString -> Word8 -> ByteStringSource
O(n\c)/ Append a byte to the end of a ByteString
append :: ByteString -> ByteString -> ByteStringSource
O(n\c)/ Append two ByteStrings
head :: ByteString -> Word8Source
O(1) Extract the first element of a ByteString, which must be non-empty.
uncons :: ByteString -> Maybe (Word8, ByteString)Source
O(1) Extract the head and tail of a ByteString, returning Nothing if it is empty.
last :: ByteString -> Word8Source
O(n\c)/ Extract the last element of a ByteString, which must be finite and non-empty.
tail :: ByteString -> ByteStringSource
O(1) Extract the elements after the head of a ByteString, which must be non-empty.
init :: ByteString -> ByteStringSource
O(n\c)/ Return all the elements of a ByteString
except the last one.
null :: ByteString -> BoolSource
O(1) Test whether a ByteString is empty.
Transforming ByteStrings
map :: (Word8 -> Word8) -> ByteString -> ByteStringSource
O(n) map
f xs
is the ByteString obtained by applying f
to each
element of xs
.
reverse :: ByteString -> ByteStringSource
O(n) reverse
xs
returns the elements of xs
in reverse order.
intersperse :: Word8 -> ByteString -> ByteStringSource
The intersperse
function takes a Word8
and a ByteString
and
`intersperses' that byte between the elements of the ByteString
.
It is analogous to the intersperse function on Lists.
intercalate :: ByteString -> [ByteString] -> ByteStringSource
O(n) The intercalate
function takes a ByteString
and a list of
ByteString
s and concatenates the list after interspersing the first
argument between each element of the list.
transpose :: [ByteString] -> [ByteString]Source
The transpose
function transposes the rows and columns of its
ByteString
argument.
Reducing ByteString
s (folds)
foldl :: (a -> Word8 -> a) -> a -> ByteString -> aSource
foldl
, applied to a binary operator, a starting value (typically
the left-identity of the operator), and a ByteString, reduces the
ByteString using the binary operator, from left to right.
foldl' :: (a -> Word8 -> a) -> a -> ByteString -> aSource
'foldl\'' is like foldl
, but strict in the accumulator.
foldl1' :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8Source
'foldl1\'' is like foldl1
, but strict in the accumulator.
foldr :: (Word8 -> a -> a) -> a -> ByteString -> aSource
foldr
, applied to a binary operator, a starting value
(typically the right-identity of the operator), and a ByteString,
reduces the ByteString using the binary operator, from right to left.
foldr1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8Source
foldr1
is a variant of foldr
that has no starting value argument,
and thus must be applied to non-empty ByteString
s
Special folds
concat :: [ByteString] -> ByteStringSource
O(n) Concatenate a list of ByteStrings.
concatMap :: (Word8 -> ByteString) -> ByteString -> ByteStringSource
Map a function over a ByteString
and concatenate the results
any :: (Word8 -> Bool) -> ByteString -> BoolSource
O(n) Applied to a predicate and a ByteString, any
determines if
any element of the ByteString
satisfies the predicate.
all :: (Word8 -> Bool) -> ByteString -> BoolSource
O(n) Applied to a predicate and a ByteString
, all
determines
if all elements of the ByteString
satisfy the predicate.
maximum :: ByteString -> Word8Source
O(n) maximum
returns the maximum value from a ByteString
minimum :: ByteString -> Word8Source
O(n) minimum
returns the minimum value from a ByteString
Building ByteStrings
Scans
scanl :: (Word8 -> Word8 -> Word8) -> Word8 -> ByteString -> ByteStringSource
Accumulating maps
mapAccumL :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)Source
mapAccumR :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)Source
Infinite ByteStrings
repeat :: Word8 -> ByteStringSource
is an infinite ByteString, with repeat
xx
the value of every
element.
replicate :: Int64 -> Word8 -> ByteStringSource
O(n)
is a ByteString of length replicate
n xn
with x
the value of every element.
cycle :: ByteString -> ByteStringSource
cycle
ties a finite ByteString into a circular one, or equivalently,
the infinite repetition of the original ByteString.
iterate :: (Word8 -> Word8) -> Word8 -> ByteStringSource
returns an infinite ByteString of repeated applications
of iterate
f xf
to x
:
iterate f x == [x, f x, f (f x), ...]
Unfolding ByteStrings
unfoldr :: (a -> Maybe (Word8, a)) -> a -> ByteStringSource
O(n) The unfoldr
function is analogous to the List 'unfoldr'.
unfoldr
builds a ByteString from a seed value. The function takes
the element and returns Nothing
if it is done producing the
ByteString or returns Just
(a,b)
, in which case, a
is a
prepending to the ByteString and b
is used as the next element in a
recursive call.
Substrings
Breaking strings
take :: Int64 -> ByteString -> ByteStringSource
drop :: Int64 -> ByteString -> ByteStringSource
splitAt :: Int64 -> ByteString -> (ByteString, ByteString)Source
takeWhile :: (Word8 -> Bool) -> ByteString -> ByteStringSource
takeWhile
, applied to a predicate p
and a ByteString xs
,
returns the longest prefix (possibly empty) of xs
of elements that
satisfy p
.
dropWhile :: (Word8 -> Bool) -> ByteString -> ByteStringSource
span :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source
break :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source
group :: ByteString -> [ByteString]Source
The group
function takes a ByteString and returns a list of
ByteStrings such that the concatenation of the result is equal to the
argument. Moreover, each sublist in the result contains only equal
elements. For example,
group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]
It is a special case of groupBy
, which allows the programmer to
supply their own equality test.
groupBy :: (Word8 -> Word8 -> Bool) -> ByteString -> [ByteString]Source
inits :: ByteString -> [ByteString]Source
O(n) Return all initial segments of the given ByteString
, shortest first.
tails :: ByteString -> [ByteString]Source
O(n) Return all final segments of the given ByteString
, longest first.
Breaking into many substrings
split :: Word8 -> ByteString -> [ByteString]Source
O(n) Break a ByteString
into pieces separated by the byte
argument, consuming the delimiter. I.e.
split '\n' "a\nb\nd\ne" == ["a","b","d","e"] split 'a' "aXaXaXa" == ["","X","X","X",""] split 'x' "x" == ["",""]
and
intercalate [c] . split c == id split == splitWith . (==)
As for all splitting functions in this library, this function does
not copy the substrings, it just constructs new ByteStrings
that
are slices of the original.
splitWith :: (Word8 -> Bool) -> ByteString -> [ByteString]Source
O(n) Splits a ByteString
into components delimited by
separators, where the predicate returns True for a separator element.
The resulting components do not contain the separators. Two adjacent
separators result in an empty component in the output. eg.
splitWith (=='a') "aabbaca" == ["","","bb","c",""] splitWith (=='a') [] == []
Predicates
isPrefixOf :: ByteString -> ByteString -> BoolSource
O(n) The isPrefixOf
function takes two ByteStrings and returns True
iff the first is a prefix of the second.
isSuffixOf :: ByteString -> ByteString -> BoolSource
O(n) The isSuffixOf
function takes two ByteStrings and returns True
iff the first is a suffix of the second.
The following holds:
isSuffixOf x y == reverse x `isPrefixOf` reverse y
Search for arbitrary substrings
Searching ByteStrings
Searching by equality
elem :: Word8 -> ByteString -> BoolSource
O(n) elem
is the ByteString
membership predicate.
Searching with a predicate
filter :: (Word8 -> Bool) -> ByteString -> ByteStringSource
O(n) filter
, applied to a predicate and a ByteString,
returns a ByteString containing those characters that satisfy the
predicate.
partition :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source
O(n) The partition
function takes a predicate a ByteString and returns
the pair of ByteStrings with elements which do and do not satisfy the
predicate, respectively; i.e.,
partition p bs == (filter p xs, filter (not . p) xs)
Indexing ByteStrings
index :: ByteString -> Int64 -> Word8Source
O(c) ByteString
index (subscript) operator, starting from 0.
elemIndex :: Word8 -> ByteString -> Maybe Int64Source
O(n) The elemIndex
function returns the index of the first
element in the given ByteString
which is equal to the query
element, or Nothing
if there is no such element.
This implementation uses memchr(3).
elemIndices :: Word8 -> ByteString -> [Int64]Source
O(n) The elemIndices
function extends elemIndex
, by returning
the indices of all elements equal to the query element, in ascending order.
This implementation uses memchr(3).
findIndex :: (Word8 -> Bool) -> ByteString -> Maybe Int64Source
The findIndex
function takes a predicate and a ByteString
and
returns the index of the first element in the ByteString
satisfying the predicate.
findIndices :: (Word8 -> Bool) -> ByteString -> [Int64]Source
The findIndices
function extends findIndex
, by returning the
indices of all elements satisfying the predicate, in ascending order.
count :: Word8 -> ByteString -> Int64Source
count returns the number of times its argument appears in the ByteString
count = length . elemIndices
But more efficiently than using length on the intermediate list.
Zipping and unzipping ByteStrings
zip :: ByteString -> ByteString -> [(Word8, Word8)]Source
zipWith :: (Word8 -> Word8 -> a) -> ByteString -> ByteString -> [a]Source
unzip :: [(Word8, Word8)] -> (ByteString, ByteString)Source
Ordered ByteStrings
Low level conversions
Copying ByteStrings
copy :: ByteString -> ByteStringSource
O(n) Make a copy of the ByteString
with its own storage.
This is mainly useful to allow the rest of the data pointed
to by the ByteString
to be garbage collected, for example
if a large string has been read in, and only a small part of it
is needed in the rest of the program.
I/O with ByteString
s
Standard input and output
getContents :: IO ByteStringSource
getContents. Equivalent to hGetContents stdin. Will read lazily
putStr :: ByteString -> IO ()Source
Write a ByteString to stdout
putStrLn :: ByteString -> IO ()Source
Write a ByteString to stdout, appending a newline byte
interact :: (ByteString -> ByteString) -> IO ()Source
The interact function takes a function of type ByteString -> ByteString
as its argument. The entire input from the standard input device is passed
to this function as its argument, and the resulting string is output on the
standard output device.
Files
readFile :: FilePath -> IO ByteStringSource
Read an entire file lazily into a ByteString
.
The Handle will be held open until EOF is encountered.
writeFile :: FilePath -> ByteString -> IO ()Source
Write a ByteString
to a file.
appendFile :: FilePath -> ByteString -> IO ()Source
Append a ByteString
to a file.
I/O with Handles
hGetContents :: Handle -> IO ByteStringSource
Read entire handle contents lazily into a ByteString
. Chunks
are read on demand, using the default chunk size.
Once EOF is encountered, the Handle is closed.
Note: the Handle
should be placed in binary mode with
hSetBinaryMode
for hGetContents
to
work correctly.
hGet :: Handle -> Int -> IO ByteStringSource
Read n
bytes into a ByteString
, directly from the specified Handle
.
hGetNonBlocking :: Handle -> Int -> IO ByteStringSource
hGetNonBlocking is similar to hGet
, except that it will never block
waiting for data to become available, instead it returns only whatever data
is available. If there is no data available to be read, hGetNonBlocking
returns empty
.
Note: on Windows and with Haskell implementation other than GHC, this
function does not work correctly; it behaves identically to hGet
.
hPut :: Handle -> ByteString -> IO ()Source
Outputs a ByteString
to the specified Handle
.
hPutNonBlocking :: Handle -> ByteString -> IO ByteStringSource
Similar to hPut
except that it will never block. Instead it returns
any tail that did not get written. This tail may be empty
in the case that
the whole string was written, or the whole original string if nothing was
written. Partial writes are also possible.
Note: on Windows and with Haskell implementation other than GHC, this
function does not work correctly; it behaves identically to hPut
.