===============
Troubleshooting
===============

Where to start diagnosing
=========================

Given the large number of technologies involved in CAPE installation, configuration and usage, chances are high one or more of them start failing, crashing or simply *dying*. When this happens, there are a few places you should always look first, since it could help you diagnosing the real problem and avoid wasting time looking at rabbit holes. These places are (no surprise here) the logs generated by each fundamental service involved in CAPE's execution. While all of them can be checked in their corresponding directories, `journalctl <https://man7.org/linux/man-pages/man1/journalctl.1.html>`_ can be leveraged to check all of them at once.

.. note::
    Please note the following errors are just random examples used to show how and where to start dealing with CAPE problems. You may never face these errors at all!

Regardless of the error you are facing there are two places where you should start looking: (1) CAPE's logs (naming inherited from cuckoo) and (2) virtqemu logs (assuming you have :ref:`installed KVM/QEMU using the kvm_qemu.sh script <installation_kvm>`.)

For example, lets consider the following situation:

    **CAPE submission (either via web or <submit.py>) apparently works, but the VM is never spawned (and analysis never launched).**

Without prior indication, the first place to check are CAPE logs, located in ``/opt/CAPEv2/log/`` (installation directory). The logs corresponding to the analyses are rotated daily, and the current log is ``cuckoo.log``. Checking the contents of ``/opt/CAPEv2/log/cuckoo.log`` one could easily spot the culprit:

.. error::
    *Error example in <cuckoo.log> file.*

    .. image:: ../../_images/screenshots/troubleshooting_error_0.png
        :align: center

Furthermore, the error states something about ``libvirt``. This is a clear indication that the corresponding logs must also be inspected. In this case, the logs of ``libvirt`` are stored under the ``virtqemud`` service. **Whenever something seems wrong regarding the virtual machines, this is the place to look after**. It can be inspected with ``$ sudo journalctl -u virtqemud -r``:

.. error::
    *Error examples by inspecting vertqemud logs with journalctl.*

    .. image:: ../../_images/screenshots/troubleshooting_error_1.png
        :align: center  

    .. image:: ../../_images/screenshots/troubleshooting_error_2.png
        :align: center
    
Additionally, you should always try to see if you are able to manually replicate the error in order to discard technologies and find out which one is failing. Considering the scenario above, when trying to manually spawn the virtual machines:

.. error::
    *Error example in Virtual Manager - KVM.*

    .. image:: ../../_images/screenshots/troubleshooting_error_3.png
        :align: center  


No Internet connection in the guest
===================================
There are reasons galore why your guest VM has no Internet connection when an analysis is fired up. Before digging into this problem, please make sure you followed the steps at :ref:`Network Configuration` to set up both the virtual machine and its connections. Furthermore, you should read the :ref:`routing` chapter in order to know and understand the different routing modes as well as the :ref:`rooter` chapter to understand what the ``Rooter`` is. 

Some considerations:

1. ``dirtyline`` should be the **interface that provides your host internet connection** like **eno1**, not a virtual interface like *virbr1*. This must be configured in the ``routing.conf`` configuration file.
2. Check ``agent.py`` is running with elevated privileges within the guest VM. 
3. Make sure you specify the correct **STATIC** IP in ``kvm.conf``.
4. Make sure you specified the correct interface in ``auxiliary.conf``.

Also, bear in mind there are already created (and some of them solved) issues about this particular problem. For example:

* https://github.com/kevoreilly/CAPEv2/issues/1234
* https://github.com/kevoreilly/CAPEv2/issues/1245
* https://github.com/kevoreilly/CAPEv2/issues/371
* https://github.com/kevoreilly/CAPEv2/issues/367
* https://github.com/kevoreilly/CAPEv2/issues/136

PCAP Generation
===============

If you are facing problems related to either tcpdump or the PCAP generation, take a look at `this issue (#1234) <https://github.com/kevoreilly/CAPEv2/issues/1234>`_.

.. note::

    Make sure the ``pcap`` group exists in your system and that the user you use to launch CAPE (presumably the `cape` user) belongs to it as well as the ``tcpdump`` binary.

Make sure the correct path is specified in ``auxiliary.conf`` for ``tcpdump``. Check the path of your local installation of tcpdump with::

    $ whereis tcpdump

Check permissions of ``tcpdump`` binary. ``cape`` user must be able to run it. Also check whether you specified the correct interface in ``auxiliary.conf``.

If you are still facing problems and the PCAP is not generating, verify the ``tcpdump`` binary belongs to the ``pcap`` group and it has the neede capabilities::

    $ sudo chgrp pcap /usr/bin/tcpdump
    $ sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/tcpdump

Other issues about this problem:

* https://github.com/kevoreilly/CAPEv2/issues/1193
