= Tailing
keywords: queue, java, tailing
author: Julia Gustafsson
:reftext: Tailing
:navtitle: Tailing
:source-highlighter: highlight.js

Reading the queue follows the same pattern as writing, except there is a possibility there is not a message when you attempt to read it.

Unlike other Java queuing solutions, messages are not lost when they are read with a tailer. This is covered in more detail in the section below on "Reading from a queue using a tailer".

.Start Reading
[source, java]
----
try (ChronicleQueue queue = ChronicleQueue.singleBuilder(path + "/trades").build()) {
   final ExcerptTailer tailer = queue.createTailer();
}
----

You can turn each message into a method call based on the content of the message, and have Chronicle Queue automatically deserialize the method arguments. Calling `reader.readOne()` will automatically skip over (filter out) any messages that do not match your method reader.

[source, java]
----
// reading using method calls
RiskMonitor monitor = System.out::println;
MethodReader reader = tailer.methodReader(monitor);
// read one message
assertTrue(reader.readOne());
----

You can decode the message yourself.

NOTE: The names, type, and order of the fields doesn't have to match.

[source, java]
----
assertTrue(tailer.readDocument(w -> w.read("trade").marshallable(
        m -> {
            LocalDateTime timestamp = m.read("timestamp").dateTime();
            String symbol = m.read("symbol").text();
            double price = m.read("price").float64();
            double quantity = m.read("quantity").float64();
            Side side = m.read("side").object(Side.class);
            String trader = m.read("trader").text();
            // do something with values.
        })));
----

You can read self-describing data values. This will check the types are correct, and convert as required.

[source, java]
----
assertTrue(tailer.readDocument(w -> {
    ValueIn in = w.getValueIn();
    int num = in.int32();
    long num2 = in.int64();
    String text = in.text();
    // do something with values
}));
----

You can read raw data as primitives and strings.

[source, java]
----
assertTrue(tailer.readBytes(in -> {
    int code = in.readByte();
    int num = in.readInt();
    long num2 = in.readLong();
    String text = in.readUtf8();
    assertEquals("Hello World", text);
    // do something with values
}));
----

or, you can get the underlying memory address and access the native memory.

[source, java]
----
assertTrue(tailer.readBytes(b -> {
    long address = b.address(b.readPosition());
    Unsafe unsafe = UnsafeMemory.UNSAFE;
    int code = unsafe.getByte(address);
    address++;
    int num = unsafe.getInt(address);
    address += 4;
    long num2 = unsafe.getLong(address);
    address += 8;
    int length = unsafe.getByte(address);
    address++;
    byte[] bytes = new byte[length];
    unsafe.copyMemory(null, address, bytes, Jvm.arrayByteBaseOffset(), bytes.length);
    String text = new String(bytes, StandardCharsets.UTF_8);
    assertEquals("Hello World", text);
    // do something with values
}));
----

NOTE: Every tailer sees every message.

An abstraction can be added to filter messages, or assign messages to just one message processor. However, in general you only need one main tailer for a topic, with possibly, some supporting tailers for monitoring etc.

As Chronicle Queue doesn't partition its topics, you get total ordering of all messages within that topic. Across topics, there is no guarantee of ordering; if you want to replay deterministically from a system which consumes from multiple topics, we suggest replaying from that system's output.

== Tailers and File Handlers Clean Up

Chronicle Queue tailers may create file handlers, the file handlers are cleaned up whenever the associated chronicle queue's `close()` method is invoked or whenever the Jvm runs a Garbage Collection.
If you are writing your code not have GC pauses and you explicitly want to clean up the file handlers, you can call the following:

[source, java]
----
((StoreTailer)tailer).releaseResources()
----

=== Using `ExcerptTailer.toEnd()`

In some applications, it may be necessary to start reading from the end of the queue (e.g. in a restart scenario).
For this use-case, `ExcerptTailer` provides the `toEnd()` method.
When the tailer direction is `FORWARD` (by default, or as set by the `ExcerptTailer.direction`
method), then calling `toEnd()` will place the tailer just *after* the last existing record in the queue.
In this case, the tailer is now ready for reading any new records appended to the queue.
Until any new messages are appended to the queue, there will be no new `DocumentContext`
available for reading:

[source,java]
----
// this will be false until new messages are appended to the queue
boolean messageAvailable = tailer.toEnd().readingDocument().isPresent();
----

If it is necessary to read backwards through the queue from the end, then the tailer can be set to read backwards:

[source,java]
----
ExcerptTailer tailer = queue.createTailer();
tailer.direction(TailerDirection.BACKWARD).toEnd();
----

When reading backwards, then the `toEnd()` method will move the tailer to the last record in the queue. If the queue is not empty, then there will be a
`DocumentContext` available for reading:

[source, java]
----
// this will be true if there is at least one message in the queue
boolean messageAvailable = tailer.toEnd().direction(TailerDirection.BACKWARD).
        readingDocument().isPresent();
----

== Restartable Tailers

AKA named tailers.

It can be useful to have a tailer which continues from where it was up to on restart of the application.

[source, java]
----
try (ChronicleQueue cq = SingleChronicleQueueBuilder.binary(tmp).build()) {
    ExcerptTailer atailer = cq.createTailer("a");
    assertEquals("test 0", atailer.readText());
    assertEquals("test 1", atailer.readText());
    assertEquals("test 2", atailer.readText()); #<1>

    ExcerptTailer btailer = cq.createTailer("b");
    assertEquals("test 0", btailer.readText()); #<3>
}

try (ChronicleQueue cq = SingleChronicleQueueBuilder.binary(tmp).build()) {
    ExcerptTailer atailer = cq.createTailer("a");
    assertEquals("test 3", atailer.readText()); #<2>
    assertEquals("test 4", atailer.readText());
    assertEquals("test 5", atailer.readText());

    ExcerptTailer btailer = cq.createTailer("b");
    assertEquals("test 1", btailer.readText()); #<4>
}
----
<1> Tailer "a" last reads message 2
<2> Tailer "a" next reads message 3
<3> Tailer "b" last reads message 0
<4> Tailer "b" next reads message 1

This is from the `RestartableTailerTest` where there are two tailers, each with a unique name.
These tailers store their index within the Queue itself and this index is maintained as the tailer uses `toStart()`, `toEnd()`, `moveToIndex()` or reads a message.

NOTE: The `direction()` is not preserved across restarts, only the next index to be read.

NOTE: The index of a tailer is only progressed when the `DocumentContext.close()` is called.
If this is prevented by an error, the same message will be read on each restart.

== Command Line Tools - Reading and Writing a Chronicle Queue

Chronicle Queue stores its data in binary format, with a file extension of `cq4`:

[source, text]
----
\�@πheader∂SCQStoreÇE��»wireType∂WireTypeÊBINARYÕwritePositionèèèèß��������ƒroll∂SCQSRollÇ*���∆length¶ÄÓ6�∆format
ÎyyyyMMdd-HH≈epoch¶ÄÓ6�»indexing∂SCQSIndexingÇN��� indexCount•�ÃindexSpacingÀindex2Indexé����ß��������…lastIndexé�
���ß��������ﬂlastAcknowledgedIndexReplicatedé�����ßˇˇˇˇˇˇˇˇ»recovery∂TimedStoreRecoveryÇ���…timeStampèèèß
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
���������������������������������������������������������������������������������������������
�����������������������������������������������������������������
----

This can often be a bit difficult to read, so it is better to dump the `cq4` files as text. This can also help you fix your production issues, as it gives you the visibility as to what has been stored in the queue, and in what order.

You can dump the queue to the terminal using `net.openhft.chronicle.queue.main.DumpMain` or `net.openhft.chronicle.queue.ChronicleReaderMain`. `DumpMain` performs a simple dump to the terminal while `ChronicleReaderMain` handles more complex operations, e.g. tailing a queue. More information and examples of usage is provided on the xref:command-line:command_line_tools.adoc[Command Line Tools] page.
