Key-value Logs

Key-value is a simple structured data format that stores a collection of key and value pairs. Some applications use key-value formatted logs for logging, and you can fetch values from these logs by specifying the respective key in a log pattern. This pattern improves the search-filter capabilities and parses out the values seamlessly.

Site24x7 helps parse your key-value logs. Additionally, Site24x7 supports the combination of a normal log with key-value log patterns. Get more information from these logs using Site24x7 AppLogs.

Getting started

  1. Log in to your Site24x7 account.
  2. Download and install the Site24x7 Server monitoring agent (Windows | Linux).
  3. Go to Admin > AppLogs > Log Types > Add Log Type.

You can refer to this Adding a Log Type help document for more information.

Key-value log pattern

Sample log

date="2014-06-18 11:57:46,719" at=info method=POST path="/en/admin/post/1/edit?k0=v0&k1=v1" host=helloworld-symfony.herokuapp.com request_id=e8843b25-3587-4229-a430-c93360a0e89f fwd="121.24.53.11" dyno=web.1 connect=1ms service=243ms status=302 bytes=559 protocol=https

Log pattern

The log pattern should start with the keyword keyvalue and contain a minimum of one date field. The following is the sample pattern defined by Site24x7 to parse the above key-value logs:

keyvalue $date:date:yyyy-MM-dd HH:mm:ss,SSS$ $at$ $method$ $path$ $host$ $request_id as requestid$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$ $protocol$
If you wish to change the field name, say from request_id to requestid, use the as keyword. Also, if you want to exclude a field, you can remove it from the log pattern.

This log is separated into fields, each of which will take its respective value, and is then uploaded to Site24x7.

Field name Field value
date 2014-06-18 11:57:46,719
at info
method POST
path /en/admin/post/1/edit?k0=v0&k1=v1
host helloworld-symfony.herokuapp.com
requestid e8843b25-3587-4229-a430-c93360a0e89f
fwd 121.24.53.11
dyno web.1
connect 1ms
service 243ms
status 302
bytes 559
protocol https

 

Combination of normal logs with key-value pattern logs

Sample log

2012-02-07T09:43:06.123456+00:00 heroku[router]: at=info method=GET path="/stylesheets/dev-center/library.css" host=devcenter.heroku.com fwd="204.204.204.204" dyno=web.5 connect=1ms service=18ms status=200 bytes=13

Log pattern

The following log pattern contains a combination of normal and key-value logs:

$DateTime:date$ heroku[router]: $Message:keyvalue:$at$$method$ $path$ $host$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$$

DateTime and heroku are the field names in a normal log pattern, whereas the field names followed by the Message:keyvalue keyword follow the key-value log pattern.

Field name Field value
DateTime 2012-02-07T09:43:06.123456+00:00
at info
method GET
path /stylesheets/dev-center/library.css
host devcenter.heroku.com
fwd 204.204.204.204
dyno web.5
connect 1ms
service 18ms
status 200
bytes 13

 

Related log types

Was this document helpful?
Thanks for taking the time to share your feedback. We’ll use your feedback to improve our online help resources.