-- Hoogle documentation, generated by Haddock
-- See Hoogle, http://www.haskell.org/hoogle/


-- | In memory storage of deeply evaluated data structure
gr
grThis package provides minimal functionality for working with "compact
grregions", which hold a fully evaluated Haskell object graph. These
grregions maintain the invariant that no pointers live inside the struct
grthat point outside it, which ensures efficient garbage collection
grwithout ever reading the structure contents (effectively, it works as
gra manually managed "oldest generation" which is never freed until the
grwhole is released). Internally, the struct is stored a single
grcontiguous block of memory, which allows efficient serialization and
grdeserialization of structs for distributed computing.
@package ghc-compact
@version 0.1.0.0


-- | This module provides a data structure, called a <a>Compact</a>, for
grholding immutable, fully evaluated data in a consecutive block of
grmemory. Compact regions are good for two things:
gr
gr<ol>
gr<li>Data in a compact region is not traversed during GC; any incoming
grpointer to a compact region keeps the entire region live. Thus, if you
grput a long-lived data structure in a compact region, you may save a
grlot of cycles during major collections, since you will no longer be
gr(uselessly) retraversing this data structure.</li>
gr<li>Because the data is stored contiguously, you can easily dump the
grmemory to disk and/or send it over the network. For applications that
grare not bandwidth bound (GHC's heap representation can be as much of a
grx4 expansion over a binary serialization), this can lead to
grsubstantial speedups.</li>
gr</ol>
gr
grFor example, suppose you have a function <tt>loadBigStruct :: IO
grBigStruct</tt>, which loads a large data structure from the file
grsystem. You can "compact" the structure with the following code:
gr
gr<pre>
grdo r &lt;- <a>compact</a> =&lt;&lt; loadBigStruct
gr   let x = <a>getCompact</a> r :: BigStruct
gr   -- Do things with x
gr</pre>
gr
grNote that <a>compact</a> will not preserve internal sharing; use
gr<a>compactWithSharing</a> (which is 10x slower) if you have cycles
grand/or must preserve sharing. The <a>Compact</a> pointer <tt>r</tt>
grcan be used to add more data to a compact region; see
gr<a>compactAdd</a> or <a>compactAddWithSharing</a>.
gr
grThe implementation of compact regions is described by:
gr
gr<ul>
gr<li>Edward Z. Yang, Giovanni Campagna, Ömer Ağacan, Ahmed El-Hassany,
grAbhishek Kulkarni, Ryan Newton. "/Efficient communication and
grCollection with Compact Normal Forms/". In Proceedings of the 20th ACM
grSIGPLAN International Conference on Functional Programming. September
gr2015. <a>http://ezyang.com/compact.html</a></li>
gr</ul>
gr
grThis library is supported by GHC 8.2 and later.
module GHC.Compact

-- | A <a>Compact</a> contains fully evaluated, pure, immutable data.
gr
gr<a>Compact</a> serves two purposes:
gr
gr<ul>
gr<li>Data stored in a <a>Compact</a> has no garbage collection
groverhead. The garbage collector considers the whole <a>Compact</a> to
grbe alive if there is a reference to any object within it.</li>
gr<li>A <a>Compact</a> can be serialized, stored, and deserialized
gragain. The serialized data can only be deserialized by the exact
grbinary that created it, but it can be stored indefinitely before
grdeserialization.</li>
gr</ul>
gr
grCompacts are self-contained, so compacting data involves copying it;
grif you have data that lives in two <a>Compact</a>s, each will have a
grseparate copy of the data.
gr
grThe cost of compaction is similar to the cost of GC for the same data,
grbut it is performed only once. However, because "GHC.Compact.compact"
grdoes not stop-the-world, retaining internal sharing during the
grcompaction process is very costly. The user can choose whether to
gr<a>compact</a> or <a>compactWithSharing</a>.
gr
grWhen you have a <tt><a>Compact</a> a</tt>, you can get a pointer to
grthe actual object in the region using "GHC.Compact.getCompact". The
gr<a>Compact</a> type serves as handle on the region itself; you can use
grthis handle to add data to a specific <a>Compact</a> with
gr<a>compactAdd</a> or <a>compactAddWithSharing</a> (giving you a new
grhandle which corresponds to the same compact region, but points to the
grnewly added object in the region). At the moment, due to technical
grreasons, it's not possible to get the <tt><a>Compact</a> a</tt> if you
gronly have an <tt>a</tt>, so make sure you hold on to the handle as
grnecessary.
gr
grData in a compact doesn't ever move, so compacting data is also a way
grto pin arbitrary data structures in memory.
gr
grThere are some limitations on what can be compacted:
gr
gr<ul>
gr<li>Functions. Compaction only applies to data.</li>
gr<li>Pinned <a>ByteArray#</a> objects cannot be compacted. This is for
gra good reason: the memory is pinned so that it can be referenced by
graddress (the address might be stored in a C data structure, for
grexample), so we can't make a copy of it to store in the
gr<a>Compact</a>.</li>
gr<li>Objects with mutable pointer fields (e.g. <a>IORef</a>,
gr<a>MutableArray</a>) also cannot be compacted, because subsequent
grmutation would destroy the property that a compact is
grself-contained.</li>
gr</ul>
gr
grIf compaction encounters any of the above, a <tt>CompactionFailed</tt>
grexception will be thrown by the compaction operation.
data Compact a
Compact :: Compact# -> a -> (MVar ()) -> Compact a

-- | Compact a value. <i>O(size of unshared data)</i>
gr
grIf the structure contains any internal sharing, the shared data will
grbe duplicated during the compaction process. This will not terminate
grif the structure contains cycles (use <a>compactWithSharing</a>
grinstead).
gr
grThe object in question must not contain any functions or data with
grmutable pointers; if it does, <a>compact</a> will raise an exception.
grIn the future, we may add a type class which will help statically
grcheck if this is the case or not.
compact :: a -> IO (Compact a)

-- | Compact a value, retaining any internal sharing and cycles. <i>O(size
grof data)</i>
gr
grThis is typically about 10x slower than <a>compact</a>, because it
grworks by maintaining a hash table mapping uncompacted objects to
grcompacted objects.
gr
grThe object in question must not contain any functions or data with
grmutable pointers; if it does, <a>compact</a> will raise an exception.
grIn the future, we may add a type class which will help statically
grcheck if this is the case or not.
compactWithSharing :: a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>. This will help you avoid
grcopying when the value contains pointers into the compact region, but
grremember that after compaction this value will only be deallocated
grwith the entire compact region.
gr
grBehaves exactly like <a>compact</a> with respect to sharing and what
grdata it accepts.
compactAdd :: Compact b -> a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>, like <a>compactAdd</a>, but
grbehaving exactly like <a>compactWithSharing</a> with respect to
grsharing and what data it accepts.
compactAddWithSharing :: Compact b -> a -> IO (Compact a)

-- | Retrieve a direct pointer to the value pointed at by a <a>Compact</a>
grreference. If you have used <a>compactAdd</a>, there may be multiple
gr<a>Compact</a> references into the same compact region. Upholds the
grproperty:
gr
gr<pre>
grinCompact c (getCompact c) == True
gr</pre>
getCompact :: Compact a -> a

-- | Check if the second argument is inside the passed <a>Compact</a>.
inCompact :: Compact b -> a -> IO Bool

-- | Check if the argument is in any <a>Compact</a>. If true, the value in
grquestion is also fully evaluated, since any value in a compact region
grmust be fully evaluated.
isCompact :: a -> IO Bool

-- | Returns the size in bytes of the compact region.
compactSize :: Compact a -> IO Word

-- | <ul>
gr<li>Experimental.* This function doesn't actually resize a compact
grregion; rather, it changes the default block size which we allocate
grwhen the current block runs out of space, and also appends a block to
grthe compact region.</li>
gr</ul>
compactResize :: Compact a -> Word -> IO ()

-- | Make a new <a>Compact</a> object, given a pointer to the true
grunderlying region. You must uphold the invariant that <tt>a</tt> lives
grin the compact region.
mkCompact :: Compact# -> a -> State# RealWorld -> (# State# RealWorld, Compact a #)

-- | Transfer <tt>a</tt> into a new compact region, with a preallocated
grsize, possibly preserving sharing or not. If you know how big the data
grstructure in question is, you can save time by picking an appropriate
grblock size for the compact region.
compactSized :: Int -> Bool -> a -> IO (Compact a)


-- | This module contains support for serializing a Compact for network
grtransmission and on-disk storage.
gr
gr<i>Since: 1.0.0</i>
module GHC.Compact.Serialized

-- | A serialized version of the <a>Compact</a> metadata (each block with
graddress and size and the address of the root). This structure is meant
grto be sent alongside the actual <a>Compact</a> data. It can be sent
grout of band in advance if the data is to be sent over RDMA (which
grrequires both sender and receiver to have pinned buffers).
data SerializedCompact a
SerializedCompact :: [(Ptr a, Word)] -> Ptr a -> SerializedCompact a
[serializedCompactBlockList] :: SerializedCompact a -> [(Ptr a, Word)]
[serializedCompactRoot] :: SerializedCompact a -> Ptr a

-- | Serialize the <a>Compact</a>, and call the provided function with with
grthe <a>Compact</a> serialized representation. It is not safe to return
grthe pointer from the action and use it after the action completes: all
gruses must be inside this bracket, since we cannot guarantee that the
grcompact region will stay live from the <a>Ptr</a> object. For example,
grit would be unsound to use <tt>unsafeInterleaveIO</tt> to lazily
grconstruct a lazy bytestring from the <a>Ptr</a>.
withSerializedCompact :: Compact a -> (SerializedCompact a -> IO c) -> IO c

-- | Deserialize a <a>SerializedCompact</a> into a in-memory
gr<a>Compact</a>. The provided function will be called with the address
grand size of each newly allocated block in succession, and should fill
grthe memory from the external source (eg. by reading from a socket or
grfrom disk) <a>importCompact</a> can return Nothing if the
gr<a>Compact</a> was corrupt or it had pointers that could not be
gradjusted.
importCompact :: SerializedCompact a -> (Ptr b -> Word -> IO ()) -> IO (Maybe (Compact a))

-- | Convenience function for importing a compact region that is
grrepresented by a list of strict <tt>ByteString</tt>s.
importCompactByteStrings :: SerializedCompact a -> [ByteString] -> IO (Maybe (Compact a))
