-- Hoogle documentation, generated by Haddock
-- See Hoogle, http://www.haskell.org/hoogle/


-- | In memory storage of deeply evaluated data structure
--   
--   This package provides minimal functionality for working with "compact
--   regions", which hold a fully evaluated Haskell object graph. These
--   regions maintain the invariant that no pointers live inside the struct
--   that point outside it, which ensures efficient garbage collection
--   without ever reading the structure contents (effectively, it works as
--   a manually managed "oldest generation" which is never freed until the
--   whole is released). Internally, the struct is stored a single
--   contiguous block of memory, which allows efficient serialization and
--   deserialization of structs for distributed computing.
@package ghc-compact
@version 0.1.0.0


-- | This module provides a data structure, called a <a>Compact</a>, for
--   holding immutable, fully evaluated data in a consecutive block of
--   memory. Compact regions are good for two things:
--   
--   <ol>
--   <li>Data in a compact region is not traversed during GC; any incoming
--   pointer to a compact region keeps the entire region live. Thus, if you
--   put a long-lived data structure in a compact region, you may save a
--   lot of cycles during major collections, since you will no longer be
--   (uselessly) retraversing this data structure.</li>
--   <li>Because the data is stored contiguously, you can easily dump the
--   memory to disk and/or send it over the network. For applications that
--   are not bandwidth bound (GHC's heap representation can be as much of a
--   x4 expansion over a binary serialization), this can lead to
--   substantial speedups.</li>
--   </ol>
--   
--   For example, suppose you have a function <tt>loadBigStruct :: IO
--   BigStruct</tt>, which loads a large data structure from the file
--   system. You can "compact" the structure with the following code:
--   
--   <pre>
--   do r &lt;- <a>compact</a> =&lt;&lt; loadBigStruct
--      let x = <a>getCompact</a> r :: BigStruct
--      -- Do things with x
--   </pre>
--   
--   Note that <a>compact</a> will not preserve internal sharing; use
--   <a>compactWithSharing</a> (which is 10x slower) if you have cycles
--   and/or must preserve sharing. The <a>Compact</a> pointer <tt>r</tt>
--   can be used to add more data to a compact region; see
--   <a>compactAdd</a> or <a>compactAddWithSharing</a>.
--   
--   The implementation of compact regions is described by:
--   
--   <ul>
--   <li>Edward Z. Yang, Giovanni Campagna, Ömer Ağacan, Ahmed El-Hassany,
--   Abhishek Kulkarni, Ryan Newton. "/Efficient communication and
--   Collection with Compact Normal Forms/". In Proceedings of the 20th ACM
--   SIGPLAN International Conference on Functional Programming. September
--   2015. <a>http://ezyang.com/compact.html</a></li>
--   </ul>
--   
--   This library is supported by GHC 8.2 and later.
module GHC.Compact

-- | A <a>Compact</a> contains fully evaluated, pure, immutable data.
--   
--   <a>Compact</a> serves two purposes:
--   
--   <ul>
--   <li>Data stored in a <a>Compact</a> has no garbage collection
--   overhead. The garbage collector considers the whole <a>Compact</a> to
--   be alive if there is a reference to any object within it.</li>
--   <li>A <a>Compact</a> can be serialized, stored, and deserialized
--   again. The serialized data can only be deserialized by the exact
--   binary that created it, but it can be stored indefinitely before
--   deserialization.</li>
--   </ul>
--   
--   Compacts are self-contained, so compacting data involves copying it;
--   if you have data that lives in two <a>Compact</a>s, each will have a
--   separate copy of the data.
--   
--   The cost of compaction is fully evaluating the data + copying it.
--   However, because <a>compact</a> does not stop-the-world, retaining
--   internal sharing during the compaction process is very costly. The
--   user can choose whether to <a>compact</a> or
--   <a>compactWithSharing</a>.
--   
--   When you have a <tt><a>Compact</a> a</tt>, you can get a pointer to
--   the actual object in the region using <a>getCompact</a>. The
--   <a>Compact</a> type serves as handle on the region itself; you can use
--   this handle to add data to a specific <a>Compact</a> with
--   <a>compactAdd</a> or <a>compactAddWithSharing</a> (giving you a new
--   handle which corresponds to the same compact region, but points to the
--   newly added object in the region). At the moment, due to technical
--   reasons, it's not possible to get the <tt><a>Compact</a> a</tt> if you
--   only have an <tt>a</tt>, so make sure you hold on to the handle as
--   necessary.
--   
--   Data in a compact doesn't ever move, so compacting data is also a way
--   to pin arbitrary data structures in memory.
--   
--   There are some limitations on what can be compacted:
--   
--   <ul>
--   <li>Functions. Compaction only applies to data.</li>
--   <li>Pinned <a>ByteArray#</a> objects cannot be compacted. This is for
--   a good reason: the memory is pinned so that it can be referenced by
--   address (the address might be stored in a C data structure, for
--   example), so we can't make a copy of it to store in the
--   <a>Compact</a>.</li>
--   <li>Objects with mutable pointer fields (e.g. <a>IORef</a>,
--   <a>MutableArray</a>) also cannot be compacted, because subsequent
--   mutation would destroy the property that a compact is
--   self-contained.</li>
--   </ul>
--   
--   If compaction encounters any of the above, a <a>CompactionFailed</a>
--   exception will be thrown by the compaction operation.
data Compact a
Compact :: Compact# -> a -> MVar () -> Compact a

-- | Compact a value. <i>O(size of unshared data)</i>
--   
--   If the structure contains any internal sharing, the shared data will
--   be duplicated during the compaction process. This will not terminate
--   if the structure contains cycles (use <a>compactWithSharing</a>
--   instead).
--   
--   The object in question must not contain any functions or data with
--   mutable pointers; if it does, <a>compact</a> will raise an exception.
--   In the future, we may add a type class which will help statically
--   check if this is the case or not.
compact :: a -> IO (Compact a)

-- | Compact a value, retaining any internal sharing and cycles. <i>O(size
--   of data)</i>
--   
--   This is typically about 10x slower than <a>compact</a>, because it
--   works by maintaining a hash table mapping uncompacted objects to
--   compacted objects.
--   
--   The object in question must not contain any functions or data with
--   mutable pointers; if it does, <a>compact</a> will raise an exception.
--   In the future, we may add a type class which will help statically
--   check if this is the case or not.
compactWithSharing :: a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>. This will help you avoid
--   copying when the value contains pointers into the compact region, but
--   remember that after compaction this value will only be deallocated
--   with the entire compact region.
--   
--   Behaves exactly like <a>compact</a> with respect to sharing and what
--   data it accepts.
compactAdd :: Compact b -> a -> IO (Compact a)

-- | Add a value to an existing <a>Compact</a>, like <a>compactAdd</a>, but
--   behaving exactly like <a>compactWithSharing</a> with respect to
--   sharing and what data it accepts.
compactAddWithSharing :: Compact b -> a -> IO (Compact a)

-- | Retrieve a direct pointer to the value pointed at by a <a>Compact</a>
--   reference. If you have used <a>compactAdd</a>, there may be multiple
--   <a>Compact</a> references into the same compact region. Upholds the
--   property:
--   
--   <pre>
--   inCompact c (getCompact c) == True
--   </pre>
getCompact :: Compact a -> a

-- | Check if the second argument is inside the passed <a>Compact</a>.
inCompact :: Compact b -> a -> IO Bool

-- | Check if the argument is in any <a>Compact</a>. If true, the value in
--   question is also fully evaluated, since any value in a compact region
--   must be fully evaluated.
isCompact :: a -> IO Bool

-- | Returns the size in bytes of the compact region.
compactSize :: Compact a -> IO Word

-- | <b>Experimental</b> This function doesn't actually resize a compact
--   region; rather, it changes the default block size which we allocate
--   when the current block runs out of space, and also appends a block to
--   the compact region.
compactResize :: Compact a -> Word -> IO ()

-- | Make a new <a>Compact</a> object, given a pointer to the true
--   underlying region. You must uphold the invariant that <tt>a</tt> lives
--   in the compact region.
mkCompact :: Compact# -> a -> State# RealWorld -> (# State# RealWorld, Compact a #)

-- | Transfer <tt>a</tt> into a new compact region, with a preallocated
--   size (in bytes), possibly preserving sharing or not. If you know how
--   big the data structure in question is, you can save time by picking an
--   appropriate block size for the compact region.
compactSized :: Int -> Bool -> a -> IO (Compact a)


-- | This module contains support for serializing a Compact for network
--   transmission and on-disk storage.
--   
--   <i>Since: 1.0.0</i>
module GHC.Compact.Serialized

-- | A serialized version of the <a>Compact</a> metadata (each block with
--   address and size and the address of the root). This structure is meant
--   to be sent alongside the actual <a>Compact</a> data. It can be sent
--   out of band in advance if the data is to be sent over RDMA (which
--   requires both sender and receiver to have pinned buffers).
data SerializedCompact a
SerializedCompact :: [(Ptr a, Word)] -> Ptr a -> SerializedCompact a
[serializedCompactBlockList] :: SerializedCompact a -> [(Ptr a, Word)]
[serializedCompactRoot] :: SerializedCompact a -> Ptr a

-- | Serialize the <a>Compact</a>, and call the provided function with with
--   the <a>Compact</a> serialized representation. It is not safe to return
--   the pointer from the action and use it after the action completes: all
--   uses must be inside this bracket, since we cannot guarantee that the
--   compact region will stay live from the <a>Ptr</a> object. For example,
--   it would be unsound to use <tt>unsafeInterleaveIO</tt> to lazily
--   construct a lazy bytestring from the <a>Ptr</a>.
withSerializedCompact :: Compact a -> (SerializedCompact a -> IO c) -> IO c

-- | Deserialize a <a>SerializedCompact</a> into a in-memory
--   <a>Compact</a>. The provided function will be called with the address
--   and size of each newly allocated block in succession, and should fill
--   the memory from the external source (eg. by reading from a socket or
--   from disk) <a>importCompact</a> can return Nothing if the
--   <a>Compact</a> was corrupt or it had pointers that could not be
--   adjusted.
importCompact :: SerializedCompact a -> (Ptr b -> Word -> IO ()) -> IO (Maybe (Compact a))

-- | Convenience function for importing a compact region that is
--   represented by a list of strict <tt>ByteString</tt>s.
importCompactByteStrings :: SerializedCompact a -> [ByteString] -> IO (Maybe (Compact a))
