Suspicious User Agent

Detects suspicious malformed user agent strings in proxy logs

Rule Content

- title: Suspicious User Agent
  id: 7195a772-4b3f-43a4-a210-6a003d65caa1
  status: experimental
  description: Detects suspicious malformed user agent strings in proxy logs
  references:
  - https://github.com/fastly/waf_testbed/blob/master/templates/default/scanners-user-agents.data.erb
  author: Florian Roth
  logsource:
    category: proxy
    product: null
    service: null
  detection:
    selection:
      c-useragent:
      - user-agent
      - '* (compatible;MSIE *'
      - '*.0;Windows NT *'
      - Mozilla/3.0 *
      - Mozilla/2.0 *
      - Mozilla/1.0 *
      - Mozilla *
      - ' Mozilla/*'
      - Mozila/*
      - _
      - CertUtil URL Agent
      - Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0)
      - Mozilla/5.0 (Windows NT 6.3; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
      - HTTPS
    falsepositives:
      c-useragent:
      - Mozilla/3.0 * Acrobat *
    condition: selection and not falsepositives
  fields:
  - ClientIP
  - c-uri
  - c-useragent
  falsepositives:
  - Unknown
  level: high

Querying Elasticsearch

Import Libraries



In [ ]:

    
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search
import pandas as pd

Initialize Elasticsearch client



In [ ]:

    
es = Elasticsearch(['http://helk-elasticsearch:9200'])
searchContext = Search(using=es, index='logs-*', doc_type='doc')

Run Elasticsearch Query



In [ ]:

    
s = searchContext.query('query_string', query='(c-useragent.keyword:(user\-agent OR *\ \(compatible;MSIE\ * OR *.0;Windows\ NT\ * OR Mozilla\/3.0\ * OR Mozilla\/2.0\ * OR Mozilla\/1.0\ * OR Mozilla\ * OR \ Mozilla\/* OR Mozila\/* OR _ OR CertUtil\ URL\ Agent OR Mozilla\/5.0\ \(Windows\ NT\ 10.0;\ Win64;\ x64;\ rv\:60.0\) OR Mozilla\/5.0\ \(Windows\ NT\ 6.3;\ WOW64;\ rv\:28.0\)\ Gecko\/20100101\ Firefox\/28.0 OR HTTPS) AND (NOT (c-useragent.keyword:(Mozilla\/3.0\ *\ Acrobat\ *))))')
response = s.execute()
if response.success():
    df = pd.DataFrame((d.to_dict() for d in s.scan()))

Show Results



In [ ]:

    
df.head()