[[configuration-interfaces]]
=== Network Device Configuration

The `interfaces` section of the +{beatname_lc}.yml+ config file configures the sniffer. Here is an example configuration:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: any
packetbeat.interfaces.snaplen: 1514
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 100
------------------------------------------------------------------------------

==== Sniffing Options

You can specify the following options in the `interfaces` section of the +{beatname_lc}.yml+ config file:

===== device

The network device to capture traffic from. The specified device is set automatically to promiscuous mode,
meaning that Packetbeat can capture traffic from other hosts on the same LAN.

Example:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: eth0
------------------------------------------------------------------------------

On Linux, you can specify `any` for the device, and Packetbeat captures all
messages sent or received by the server where Packetbeat is installed.

NOTE: When you specify `any` for the device, the interfaces are not set
      to promiscuous mode.

The `device` option also accepts specifying the device by its index in the list of
devices available for sniffing. To obtain the list of available devices,
run Packetbeat with the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
packetbeat -devices
----------------------------------------------------------------------

This command returns a list that looks something like the following:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
0: en0 (No description available)
1: awdl0 (No description available)
2: bridge0 (No description available)
3: fw0 (No description available)
4: en1 (No description available)
5: en2 (No description available)
6: p2p0 (No description available)
7: en4 (No description available)
8: lo0 (No description available)
----------------------------------------------------------------------

The following example sets up sniffing on the
first interface in the list:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: 0
------------------------------------------------------------------------------

Specifying the index is especially useful on Windows where device names can be long.

===== snaplen

The maximum size of the packets to capture. The
default is 65535, which is large enough for almost all networks and interface
types. If you sniff on a physical network interface, the optimal setting is
the MTU size. On virtual interfaces, however, it's safer to accept the default value.

Example:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: eth0
packetbeat.interfaces.snaplen: 1514
------------------------------------------------------------------------------

===== type

Packetbeat supports three sniffer types:

 * `pcap`, which uses the libpcap library and works on most platforms, but
   it's not the fastest option.
 * `af_packet`, which uses memory-mapped sniffing. This option is faster than libpcap
   and doesn't require a kernel module, but it's Linux-specific.
 * `pf_ring`, which makes use of an ntop.org
   http://www.ntop.org/products/pf_ring/[project]. This setting provides the best
   sniffing speed, but it requires a kernel module, and it's Linux-specific.

NOTE: The `pf_ring` option requires re-compilation of Packetbeat and is not
      officially supported by Elastic at this moment.

The default sniffer type is `pcap`.

Here is an example configuration that specifies
the `af_packet` sniffing type:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: eth0
packetbeat.interfaces.type: af_packet
------------------------------------------------------------------------------

On Linux, if you are trying to optimize the CPU usage of Packetbeat, we
recommend trying the `af_packet` option. Read <<capturing-options>> for more
details.

If you use the `af_packet` sniffer, you can tune its behaviour by specifying the
following options:

===== buffer_size_mb

The maximum size of the shared memory buffer to use
between the kernel and user space. A bigger buffer usually results in lower CPU
usage, but consumes more memory. This setting is only available for the
`af_packet` sniffer type. The default is 30 MB.

Example:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: eth0
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 100
------------------------------------------------------------------------------

===== with_vlans

Packetbeat automatically generates a
https://en.wikipedia.org/wiki/Berkeley_Packet_Filter[BPF] for capturing only
the traffic on ports where it expects to find known protocols.
For example, if you have configured port 80 for HTTP and port 3306 for MySQL,
Packetbeat generates the following BPF filter: `"port 80 or port 3306"`.

However, if the traffic contains https://en.wikipedia.org/wiki/IEEE_802.1Q[VLAN]
tags, the filter that Packetbeat generates is ineffective because the
offset is moved by four bytes. To fix this, you can enable the `with_vlans` option, which
generates a BPF filter that looks like this: `"port 80 or port 3306 or (vlan and (port 80 or port 3306))"`.

===== bpf_filter

Packetbeat automatically generates a
https://en.wikipedia.org/wiki/Berkeley_Packet_Filter[BPF] for capturing only
the traffic on ports where it expects to find known protocols.
For example, if you have configured port 80 for HTTP and port 3306 for MySQL,
Packetbeat generates the following BPF filter: `"port 80 or port 3306"`.

You can use the `bpf_filter` setting to overwrite the generated BPF filter. For example:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.interfaces.device: eth0
packetbeat.interfaces.bpf_filter: "net 192.168.238.0/0 and port 80 and port 3306"
------------------------------------------------------------------------------

NOTE: This setting disables automatic generation of the BPF filter. If
you use this setting, it's your responsibility to keep the BPF filters in sync with the
ports defined in the `protocols` section.

===== ignore_outgoing

If the `ignore_outgoing` option is enabled, Packetbeat ignores all the
transactions initiated from the server running Packetbeat.

This is useful when two Packetbeat instances publish the same transactions. Because one Packetbeat
sees the transaction in its outgoing queue and the other sees it in its incoming
queue, you can end up with duplicate transactions. To remove the duplicates, you
can enable the `packetbeat.ignore_outgoing` option on one of the servers.

For example, in the following scenario, you see a 3-server architecture
where a Beat is installed on each server. t1 is the transaction exchanged between
Server1 and Server2, and t2 is the transaction between Server2 and Server3.

image:./images/option_ignore_outgoing.png[Beats Architecture]

By default, each transaction is indexed twice because Beat2
sees both transactions. So you would see the following published transactions
(when `ignore_outgoing` is false):

 - Beat1: t1
 - Beat2: t1 and t2
 - Beat3: t2

To avoid duplicates, you can force your Beats to send only the incoming
transactions and ignore the transactions created by the local server. So you would
see the following published transactions (when `ignore_outgoing` is true):

 - Beat1: none
 - Beat2: t1
 - Beat3: t2


[[configuration-flows]]
=== Flows Configuration

The `flows` section of the +{beatname_lc}.yml+ config file contains configuration options for bidirectional network
flows. If section is missing from configuration file, network flows are
disabled.

[source,yaml]
------------------------------------------------------------------------------
packetbeat.flows:
  timeout: 30s
  period: 10s
------------------------------------------------------------------------------

==== Options

You can specify the following options in the `flows` section of the +{beatname_lc}.yml+ config file:

===== enabled

Enables flows support if set to true. Set to false to disable network flows
support without having to delete or comment out the flows section. The default
value is true.

===== timeout

Timeout configures the lifetime of a flow. If no packets have been received for
a flow within the timeout time window, the flow is killed and reported. The
default value is 30s.

===== period

Configure the reporting interval. All flows are reported at the very same point
in time. Periodical reporting can be disabled by setting the value to -1. If
disabled, flows are still reported once being timed out. The default value is
10s.


[[configuration-protocols]]
=== Transaction Protocols Configuration

The `protocols` section of the +{beatname_lc}.yml+ config file contains configuration options for each supported protocol,
including common options like `enabled`, `ports`, `send_request`, `send_response`, and options that are protocol-specific.

Currently, Packetbeat supports the following protocols:

include::../../shared-protocol-list.asciidoc[]

Example configuration:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.icmp:
  enabled: true

packetbeat.protocols.dns:
  ports: [53]

packetbeat.protocols.http:
  ports: [80, 8080, 8000, 5000, 8002]

packetbeat.protocols.amqp:
  ports: [5672]

packetbeat.protocols.cassandra:
  ports: [9042]

packetbeat.protocols.memcache:
  ports: [11211]

packetbeat.protocols.mysql:
  ports: [3306]

packetbeat.protocols.redis:
  ports: [6379]

packetbeat.protocols.pgsql:
  ports: [5432]

packetbeat.protocols.thrift:
  ports: [9090]


------------------------------------------------------------------------------

==== Common Protocol Options

The following options are available for all protocols:

===== enabled

The enabled setting is a boolean setting to enable or disable protocols without having to comment out configuration sections. If set to false, the protocol is disabled.

The default value is true.

===== ports

Exception: For ICMP the option `enabled` has to be used instead.

The ports where Packetbeat will look to capture traffic for specific
protocols. Packetbeat installs a https://en.wikipedia.org/wiki/Berkeley_Packet_Filter[BPF]
filter based on the ports specified in this section.
If a packet doesn't match the filter, very little CPU is required to discard
the packet. Packetbeat also uses the ports specified here to determine which
parser to use for each packet.

[[send-request-option]]
===== send_request

If this option is enabled, the raw message of the request (`request` field) is
sent to Elasticsearch. The default is false. This option is useful when you want to
index the whole request. Note that for HTTP, the body is not included by
default, only the HTTP headers.

[[send-response-option]]
===== send_response

If this option is enabled, the raw message of the response (`response` field)
is sent to Elasticsearch. The default is false.  This option is useful when you
want to index the whole response. Note that for HTTP, the body is not included
by default, only the HTTP headers.

[[transaction-timeout-option]]
===== transaction_timeout

The per protocol transaction timeout. Expired transactions will no longer be correlated to incoming responses, but sent to Elasticsearch immediately.

==== ICMP Configuration Options

You can specify the following options in the `icmp` section of the +{beatname_lc}.yml+ config file:

===== enabled

The ICMP protocol can be enabled/disabled via this option. The default is true.

If enabled Packetbeat will generate the following BPF filter: `"icmp or icmp6"`.

==== DNS Configuration Options

The `dns` section of the +{beatname_lc}.yml+ config file specifies configuration options for the DNS protocol. The DNS protocol supports processing DNS messages on UDP. Here is a sample configuration section for DNS:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.dns:
  ports: [53]
  include_authorities: true
  include_additionals: true
------------------------------------------------------------------------------

===== include_authorities

If this option is enabled, dns.authority fields (authority resource records) are added to DNS events.
The default is false.

===== include_additionals

If this option is enabled, dns.additionals fields (additional resource records) are added to DNS events.
The default is false.

==== HTTP Configuration Options

The HTTP protocol has several specific configuration options. Here is a
sample configuration for the `http` section of the +{beatname_lc}.yml+ config file:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.http:
  ports: [80, 8080, 8000, 5000, 8002]
  hide_keywords: ["pass", "password", "passwd"]
  send_headers: ["User-Agent", "Cookie", "Set-Cookie"]
  split_coookie: true
  real_ip_header: "X-Forwarded-For"
------------------------------------------------------------------------------

===== hide_keywords

A list of query parameters that Packetbeat will automatically censor in
the transactions that it saves. The values associated with these parameters are replaced
by `'xxxxx'`. By default, no changes are made to the HTTP messages.

Packetbeat has this option because, unlike SQL traffic, which typically only contains the
hashes of the passwords, HTTP traffic may contain sensitive data. To reduce security risks,
you can configure this option to avoid sending the contents of certain HTTP POST
parameters.

WARNING: This option replaces query parameters from GET requests and top-level
parameters from POST requests. If sensitive data is encoded inside a
parameter that you don't specify here, Packetbeat cannot censor it. Also, note that if
you configure Packetbeat to save the raw request and response fields (see the <<send-request-option>>
and the <<send-response-option>> options), sensitive data may be present in those
fields.


===== redact_authorization

When this option is enabled, Packetbeat obscures the value of
`Authorization` and `Proxy-Authorization` HTTP headers, and censors
those strings in the response.

You should set this option to true for transactions that use Basic Authentication because
they may contain the base64 unencrypted username and password.

===== send_headers

A list of header names to capture and send to Elasticsearch. These
headers are placed under the `headers` dictionary in the resulting JSON.

===== send_all_headers

Instead of sending a white list of headers to Elasticsearch, you can
send all headers by setting this option to true. The default is false.

===== include_body_for

The list of content types for which Packetbeat exports the full HTTP payload. The HTTP body is available under
`http.request.body` and `http.response.body` for these Content-Types.

In addition, if <<send-response-option>> option is enabled, then the HTTP body is exported together with the HTTP
headers under `response` and if
<<send-request-option>> enabled, then `request` contains the entire HTTP message including the body.

In the following example, the HTML attachments of the HTTP responses are exported under the `response` field and under
`http.request.body` or `http.response.body`:

[source,yml]
------------------------------------------------------------------------------
packetbeat.protocols.http:
  ports: [80, 8080]
  send_response: true
  include_body_for: ["text/html"]
------------------------------------------------------------------------------



===== split_cookie

If the `Cookie` or `Set-Cookie` headers are sent, this option controls whether
they are split into individual values. For example, with this option set, an
HTTP response might result in the following JSON:

[source,json]
------------------------------------------------------------------------------
"response": {
  "code": 200,
  "headers": {
    "connection": "close",
    "content-language": "en",
    "content-type": "text/html; charset=utf-8",
    "date": "Fri, 21 Nov 2014 17:07:34 GMT",
    "server": "gunicorn/19.1.1",
    "set-cookie": { <1>
      "csrftoken": "S9ZuJF8mvIMT5CL4T1Xqn32wkA6ZSeyf",
      "expires": "Fri, 20-Nov-2015 17:07:34 GMT",
      "max-age": "31449600",
      "path": "/"
    },
    "vary": "Cookie, Accept-Language"
  },
  "phrase": "OK"
}
------------------------------------------------------------------------------

<1> Note that `set-cookie` is a map containing the cookie names as keys.

The default is false.

===== real_ip_header

The header field to extract the real IP from. This setting is useful when
you want to capture traffic behind a reverse proxy, but you want to get the geo-location
information. If this header is present and contains a valid IP addresses, the
information is used for the `real_ip` field.

===== max_message_size

If an individual HTTP message is larger than this setting (in bytes), it will be trimmed
to this size. Unless this value is very small (<1.5K), Packetbeat is able to still correctly
follow the transaction and create an event for it. The default is 10485760 (10 MB).


==== AMQP Configuration Options

The `amqp` section of the +{beatname_lc}.yml+ config file specifies configuration options for the AMQP 0.9.1
protocol. Here is a sample configuration:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.amqp:
  ports: [5672]
  max_body_length: 1000
  parse_headers: true
  parse_arguments: false
  hide_connection_information: true
------------------------------------------------------------------------------

===== max_body_length

The maximum size in bytes of the message displayed in the request or response fields.
Messages that are bigger than the specified size are truncated. Use this option to
avoid publishing huge messages when <<send-request-option>> or <<send-request-option>>
is enabled. The default is 1000 bytes.

===== parse_headers

If set to true, Packetbeat parses the additional arguments specified in the headers field
of a message. Those arguments are key-value pairs that specify information such as the
content type of the message or the message priority. The default is true.

===== parse_arguments

If set to true, Packetbeat parses the additional arguments specified in AMQP methods.
Those arguments are key-value pairs specified by the user and can be of any length.
The default is true.

===== hide_connection_information

If set to false, the connection layer methods of the protocol are also
displayed, such as the opening and closing of connections and channels by clients,
or the quality of service negotiation. The default is true.


[[configuration-cassandra]]
==== Cassandra Configuration Options

The following settings are specific to the Cassandra protocol. Here is a sample
configuration for the `cassandra` section of the +{beatname_lc}.yml+ config file:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.cassandra:
  send_request_header: true
  send_response_header: true
  compressor: "snappy"
  ignored_ops: ["SUPPORTED","OPTIONS"]
------------------------------------------------------------------------------

===== send_request_header

If this option is enabled, the raw message of the response (`cassandra_request.request_headers` field)
is sent to Elasticsearch. The default is true. enable `send_request` first before enable this option.

===== send_response_header

If this option is enabled, the raw message of the response (`cassandra_response.response_headers` field)
is included in published events. The default is true. enable `send_response` first before enable this option.

===== ignored_ops

This option indicates which Operator/Operators captured will be ignored. currently support:
`ERROR` ,`STARTUP` ,`READY` ,`AUTHENTICATE` ,`OPTIONS` ,`SUPPORTED` ,
`QUERY` ,`RESULT` ,`PREPARE` ,`EXECUTE` ,`REGISTER`  ,`EVENT` ,
`BATCH` ,`AUTH_CHALLENGE`,`AUTH_RESPONSE` ,`AUTH_SUCCESS` .

===== compressor

Configures the default compression algorithm being used to uncompress compressed frames by name. Currently only `snappy` is can be configured.
By default no compressor is configured.


==== Memcache Configuration Options

The `memcache` section of the +{beatname_lc}.yml+ config file specifies configuration options for the memcache
protocol. Here is a sample configuration section for memcache:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.memcache:
  ports: [11211]
  parseunknown: false
  maxvalues: 0
  maxbytespervalue: 100
  transaction_timeout: 200
  udptransactiontimeout: 200
------------------------------------------------------------------------------

===== parseunknown

When this option is enabled, it forces the memcache text protocol parser to accept unknown commands.

NOTE: The unknown commands MUST NOT contain a data part.

===== maxvalues

The maximum number of values to store in the message (multi-get).
All values will be base64 encoded.

The possible settings for this option are:

* `maxvalue: -1`, which stores all values (text based protocol multi-get)
* `maxvalue: 0`, which stores no values (default)
* `maxvalue: N`, which stores up to N values

===== maxbytespervalue

The maximum number of bytes to be copied for each value element.

NOTE: Values will be base64 encoded, so the actual size in the JSON document will be 4 times the value that
you specify for `maxbytespervalue`.

===== udptransactiontimeout

The transaction timeout in milliseconds. The defaults is 10000 milliseconds.

NOTE: Quiet messages in UDP binary protocol get responses only if there is an error.
The memcache protocol analyzer will wait for the number of milliseconds specified by
`udptransactiontimeout` before publishing quiet messages. Non-quiet messages or
quiet requests with an error response are published immediately.


==== MySQL and PgSQL Configuration Options

You can specify the following options in the `mysql` or `pgsql` sections of the +{beatname_lc}.yml+ config file:

===== max_rows

The maximum number of rows from the SQL message to publish to Elasticsearch. The
default is 10 rows.


===== max_row_length

The maximum length in bytes of a row from the SQL message to publish to
Elasticsearch. The default is 1024 bytes.

[[configuration-thrift]]
==== Thrift Configuration Options

The Thrift protocol has several specific configuration options. Here is a
sample configuration for the `thrift` section of the +{beatname_lc}.yml+ config file:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.thrift:
  transport_type: socket
  protocol_type: binary
  idl_files: ["tutorial.thrift", "shared.thrift"]
  string_max_size: 200
  collection_max_size: 20
  capture_reply: true
  obfuscate_strings: true
  drop_after_n_struct_fields: 100
------------------------------------------------------------------------------

===== transport_type

The Thrift transport type. Currently this option accepts the values `socket`
for TSocket, which is the default Thrift transport, and `framed` for the TFramed Thrift
transport. The default is `socket`.

===== protocol_type

The Thrift protocol type. Currently the only accepted value is `binary` for
the TBinary protocol, which is the default Thrift protocol.

===== idl_files

The Thrift interface description language (IDL) files for the service that
Packetbeat is monitoring. Providing the IDL files is optional, because the Thrift
messages contain enough information to decode them without having the IDL
files. However, providing the IDL enables Packetbeat to include parameter and
exception names.

===== string_max_size

The maximum length for strings in parameters or return values. If a string is longer than
this value, the string is automatically truncated to this length. Packetbeat adds dots
at the end of the string to mark that it was truncated. The default is 200.

===== collection_max_size

The maximum number of elements in a Thrift list, set, map, or structure. If a collection
has more elements than this value, Packetbeat captures only the
specified number of elements. Packetbeat adds a fictive last element `...` to the end
of the collection to mark that it was truncated. The default is 15.

===== capture_reply

If this option is set to false, Packetbeat decodes the method name from
the reply and simply skips the rest of the response message. This setting can be useful
for performance, disk usage, or data retention reasons. The default is true.

===== obfuscate_strings

If this option is set to true, Packetbeat replaces all strings found in method parameters,
return codes, or exception structures with the `"*"` string.

===== drop_after_n_struct_fields

The maximum number of fields that a structure can have before Packetbeat
ignores the whole transaction. This is a memory protection mechanism (so that
Packetbeat's memory doesn't grow indefinitely), so you would typically set this
to a relatively high value. The default is 500.


[[configuration-mongodb]]
==== MongoDB Configuration Options

The following settings are specific to the MongoDB protocol. Here is a sample
configuration for the `mongodb` section of the +{beatname_lc}.yml+ config file:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.protocols.mongodb:
  send_request: true
  send_response: true
  max_docs: 0
  max_doc_length: 0
------------------------------------------------------------------------------

The following two settings are useful for limiting the amount of data
Packetbeat indexes in the `response` fields.

===== max_docs

The maximum number of documents from the response to index in the `response` field. The
default is 10. You can set this to 0 to index an unlimited number of documents.

Packetbeat adds a `[...]` line at the end to signify that there were additional documents
that weren't saved because of this setting.

===== max_doc_length

The maximum number of characters in a single document indexed in the `response`
field. The default is 5000. You can set this to 0 to index an unlimited number
of characters per document.

If the document is trimmed because of this setting, Packetbeat adds the string `...`
at the end of the document.

Note that limiting documents in this way means that they are no longer correctly
formatted JSON objects.


[[configuration-processes]]
=== Monitored Processes Configuration

This section of the +{beatname_lc}.yml+ config file is optional, but configuring
the processes enables Packetbeat to show you not only the servers that the
traffic is flowing between, but also the processes. Packetbeat can even show you
the traffic between two processes running on the same host, which is particularly
useful when you have many services running on the same server. By default,
process matching is disabled.

When Packetbeat starts, and then periodically afterwards, it scans the process table for
processes that match the configuration file. For each of these processes, it
monitors which file descriptors it has opened. When a new packet is captured,
it reads the list of active TCP connections and matches the corresponding one
with the list of file descriptors.

On a Linux system, all this information is available via the `/proc`
file system, so Packetbeat doesn't need a kernel module.


NOTE: Process monitoring is currently only supported on
      Linux systems. Packetbeat automatically disables
      process monitoring when it detects other operating systems.

Example configuration:

[source,yaml]
------------------------------------------------------------------------------
packetbeat.procs:
  enabled: true
  monitored:
    - process: mysqld
      cmdline_grep: mysqld

    - process: pgsql
      cmdline_grep: postgres

    - process: nginx
      cmdline_grep: nginx

    - process: app
      cmdline_grep: gunicorn
------------------------------------------------------------------------------

==== Process Monitoring Options

You can specify the following process monitoring options in the `monitored` section
of the +{beatname_lc}.yml+ config file:

===== process

The name of the process as it will appear in the published transactions. The name
doesn't have to match the name of the executable, so feel free to choose something
more descriptive (for example,  "myapp" instead of "gunicorn").

===== cmdline_grep

The name used to identify the process at run time. When Packetbeat starts, and then
periodically afterwards, it scans the process table for
processes that match the values specified for this option. The match is done against the
process' command line as read from `/proc/<pid>/cmdline`.

include::../../../../libbeat/docs/generalconfig.asciidoc[]

include::../../../../libbeat/docs/outputconfig.asciidoc[]

include::../../../../libbeat/docs/shared-path-config.asciidoc[]

include::../../../../libbeat/docs/loggingconfig.asciidoc[]

include::./runconfig.asciidoc[]

include::../../../../libbeat/docs/processors-config.asciidoc[]


