-- Hoogle documentation, generated by Haddock
-- See Hoogle, http://www.haskell.org/hoogle/


-- | In memory storage of deeply evaluated data structure
gr
grThis package provides minimal functionality for working with "compact
grregions", which hold a fully evaluated Haskell object graph. These
grregions maintain the invariant that no pointers live inside the struct
grthat point outside it, which ensures efficient garbage collection
grwithout ever reading the structure contents (effectively, it works as
gra manually managed "oldest generation" which is never freed until the
grwhole is released). Internally, the struct is stored a single
grcontiguous block of memory, which allows efficient serialization and
grdeserialization of structs for distributed computing.
@package ghc-compact
@version 0.1.0.0


-- | This module provides a data structure, called a <a>Compact</a>, for
grholding immutable, fully evaluated data in a consecutive block of
grmemory. Compact regions are good for two things:
gr
gr<ol>
gr<li>Data in a compact region is not traversed during GC; any incoming
grpointer to a compact region keeps the entire region live. Thus, if you
grput a long-lived data structure in a compact region, you may save a
grlot of cycles during major collections, since you will no longer be
gr(uselessly) retraversing this data structure.</li>
gr<li>Because the data is stored contiguously, you can easily dump the
grmemory to disk and/or send it over the network. For applications that
grare not bandwidth bound (GHC's heap representation can be as much of a
grx4 expansion over a binary serialization), this can lead to
grsubstantial speedups.</li>
gr</ol>
gr
grFor example, suppose you have a function <tt>loadBigStruct :: IO
grBigStruct</tt>, which loads a large data structure from the file
grsystem. You can "compact" the structure with the following code:
gr
gr<pre>
grdo r &lt;- <a>compact</a> =&lt;&lt; loadBigStruct
gr   let x = <a>getCompact</a> r :: BigStruct
gr   -- Do things with x
gr</pre>
gr
grNote that <a>compact</a> will not preserve internal sharing; use
gr<a>compactWithSharing</a> (which is 10x slower) if you have cycles
grand/or must preserve sharing. The <a>Compact</a> pointer <tt>r</tt>
grcan be used to add more data to a compact region; see
gr<a>compactAdd</a> or <a>compactAddWithSharing</a>.
gr
grThe implementation of compact regions is described by:
gr
gr<ul>
gr<li>Edward Z. Yang, Giovanni Campagna, Ömer Ağacan, Ahmed El-Hassany,
grAbhishek Kulkarni, Ryan Newton. "/Efficient communication and
grCollection with Compact Normal Forms/". In Proceedings of the 20th ACM
grSIGPLAN International Conference on Functional Programming. September
gr2015. <a>http://ezyang.com/compact.html</a></li>
gr</ul>
gr
grThis library is supported by GHC 8.2 and later.
module GHC.Compact

-- | A <a>Compact</a> contains fully evaluated, pure, immutable data.
gr
gr<a>Compact</a> serves two purposes:
gr
gr<ul>
gr<li>Data stored in a <a>Compact</a> has no garbage collection
groverhead. The garbage collector considers the whole <a>Compact</a> to
grbe alive if there is a reference to any object within it.</li>
gr<li>A <a>Compact</a> can be serialized, stored, and deserialized
gragain. The serialized data can only be deserialized by the exact
grbinary that created it, but it can be stored indefinitely before
grdeserialization.</li>
gr</ul>
gr
grCompacts are self-contained, so compacting data involves copying it;
grif you have data that lives in two <a>Compact</a>s, each will have a
grseparate copy of the data.
gr
grThe cost of compaction is similar to the cost of GC for the same data,
grbut it is performed only once. However, because "GHC.Compact.compact"
grdoes not stop-the-world, retaining internal sharing during the
grcompaction process is very costly. The user can choose whether to
gr<a>compact</a> or <a>compactWithSharing</a>.
gr
grWhen you have a <tt><a>Compact</a> a</tt>, you can get a pointer to
grthe actual object in the region using "GHC.Compact.getCompact". The
gr<a>Compact</a> type serves as handle on the region itself; you can use
grthis handle to add data to a specific <a>Compact</a> with
gr<a>compactAdd</a> or <a>compactAddWithSharing</a> (giving you a new
grhandle which corresponds to the same compact region, but points to the
grnewly added object in the region). At the moment, due to technical
grreasons, it's not possible to get the <tt><a>Compact</a> a</tt> if you
gronly have an <tt>a</tt>, so make sure you hold on to the handle as
grnecessary.
gr
grData in a compact doesn't ever move, so compacting data is also a way
grto pin arbitrary data structures in memory.
gr
grThere are some limitations on what can be compacted:
gr
gr<ul>
gr<li>Functions. Compaction only applies to data.</li>
gr<li>Pinned <a>ByteArray#</a> objects cannot be compacted. This is for
gra good reason: the memory is pinned so that it can be referenced by
graddress (the address might be stored in a C data structure, for
grexample), so we can't make a copy of it to store in the
gr<a>Compact</a>.</li>
gr<li>Objects with mutable pointer fields (e.g. <a>IORef</a>,
gr<a>MutableArray</a>) also cannot be compacted, because subsequent
grmutation would destroy the property that a compact is
grself-contained.</li>
gr</ul>
gr
grIf compaction encounters any of the above, a <tt>CompactionFailed</tt>
grexception will be thrown by the compaction operation.
data Compact a
Compact :: Compact# -> a -> (MVar ()) -> Compact a

-- | Compact a value. <i>O(size of unshared data)</i>
gr
grIf the structure contains any internal sharing, the shared data will
grbe duplicated during the compaction process. This will not terminate
grif the structure contains cycles (use <a>compactWithSharing</a>
grinstead).
gr
grThe object in question must not contain any functions or data with
grmutable pointers; if it does, <a>compact</a> will raise an exception.
grIn the future, we may add a type class which will help statically
grcheck if this is the case or not.
compact :: a -> IO (Compact a)

-- | Compact a value, retaining any internal sharing and cycles. <i>O(size
grof data)</i>
gr
grThis is typically about 10x slower than <a>compact</a>, because it
grworks by maintaining a hash table mapping uncompacted objects to
grcompacted objects.
gr
grThe object in question must not contain any functions or data with
grmutable pointers; if it does, <a>compact</a> will raise an exception.
grIn the future, we may add a type class which will help statically
grcheck if this is the case or not.
compactWithSharing :: a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>. This will help you avoid
grcopying when the value contains pointers into the compact region, but
grremember that after compaction this value will only be deallocated
grwith the entire compact region.
gr
grBehaves exactly like <a>compact</a> with respect to sharing and what
grdata it accepts.
compactAdd :: Compact b -> a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>, like <a>compactAdd</a>, but
grbehaving exactly like <a>compactWithSharing</a> with respect to
grsharing and what data it accepts.
compactAddWithSharing :: Compact b -> a -> IO (Compact a)

-- | Retrieve a direct pointer to the value pointed at by a <a>Compact</a>
grreference. If you have used <a>compactAdd</a>, there may be multiple
gr<a>Compact</a> references into the same compact region. Upholds the
grproperty:
gr
gr<pre>
grinCompact c (getCompact c) == True
gr</pre>
getCompact :: Compact a -> a

-- | Check if the second argument is inside the passed <a>Compact</a>.
inCompact :: Compact b -> a -> IO Bool

-- | Check if the argument is in any <a>Compact</a>. If true, the value in
grquestion is also fully evaluated, since any value in a compact region
grmust be fully evaluated.
isCompact :: a -> IO Bool

-- | Returns the size in bytes of the compact region.
compactSize :: Compact a -> IO Word

-- | <ul>
gr<li>Experimental.* This function doesn't actually resize a compact
grregion; rather, it changes the default block size which we allocate
grwhen the current block runs out of space, and also appends a block to
grthe compact region.</li>
gr</ul>
compactResize :: Compact a -> Word -> IO ()

-- | Make a new <a>Compact</a> object, given a pointer to the true
grunderlying region. You must uphold the invariant that <tt>a</tt> lives
grin the compact region.
mkCompact :: Compact# -> a -> State# RealWorld -> (# State# RealWorld, Compact a #)

-- | Transfer <tt>a</tt> into a new compact region, with a preallocated
grsize, possibly preserving sharing or not. If you know how big the data
grstructure in question is, you can save time by picking an appropriate
grblock size for the compact region.
compactSized :: Int -> Bool -> a -> IO (Compact a)


-- | This module contains support for serializing a Compact for network
grtransmission and on-disk storage.
gr
gr<i>Since: 1.0.0</i>
module GHC.Compact.Serialized

-- | A serialized version of the <a>Compact</a> metadata (each block with
graddress and size and the address of the root). This structure is meant
grto be sent alongside the actual <a>Compact</a> data. It can be sent
grout of band in advance if the data is to be sent over RDMA (which
grrequires both sender and receiver to have pinned buffers).
data SerializedCompact a
SerializedCompact :: [(Ptr a, Word)] -> Ptr a -> SerializedCompact a
[serializedCompactBlockList] :: SerializedCompact a -> [(Ptr a, Word)]
[serializedCompactRoot] :: SerializedCompact a -> Ptr a

-- | Serialize the <a>Compact</a>, and call the provided function with with
grthe <a>Compact</a> serialized representation. It is not safe to return
grthe pointer from the action and use it after the action completes: all
gruses must be inside this bracket, since we cannot guarantee that the
grcompact region will stay live from the <a>Ptr</a> object. For example,
grit would be unsound to use <tt>unsafeInterleaveIO</tt> to lazily
grconstruct a lazy bytestring from the <a>Ptr</a>.
withSerializedCompact :: Compact a -> (SerializedCompact a -> IO c) -> IO c

-- | Deserialize a <a>SerializedCompact</a> into a in-memory
gr<a>Compact</a>. The provided function will be called with the address
grand size of each newly allocated block in succession, and should fill
grthe memory from the external source (eg. by reading from a socket or
grfrom disk) <a>importCompact</a> can return Nothing if the
gr<a>Compact</a> was corrupt or it had pointers that could not be
gradjusted.
importCompact :: SerializedCompact a -> (Ptr b -> Word -> IO ()) -> IO (Maybe (Compact a))

-- | Convenience function for importing a compact region that is
grrepresented by a list of strict <tt>ByteString</tt>s.
importCompactByteStrings :: SerializedCompact a -> [ByteString] -> IO (Maybe (Compact a))
