The Castra ELK Logger for USM Appliance

A fast and scalable solution to pair with your existing USM instance

 

 

Do you need to add long-term logging to your USM Appliance? Do you want to combine the search power of Elastic with your advanced USM platform? If you are looking to expand your raw log storage while adding new possibilities for analytics, visualization and reporting, Castra’s Elastic Logger is for you.

"ElasticSearch is fast! Based on our testing on lab and production systems, we’re seeing 50x-100x speed improvements"

Castra has developed a powerful log management tool meant to become, expand or replace your existing USM Appliance Logger. It is a fully-integrated, drop-in replacement that is built using the ultra-fast ElasticSearch engine (a standard ELK stack), but incorporates several custom components that allow it to connect transparently to your USM Appliance as if it were a "real" Logger. Treat it like any other long term Logger. It brings fully indexed, rapid search capability to your log data, plus all of the benefits of the Kibana UI for advanced reporting and visualizations.

From your USM Appliance UI, it appears like a standard Logger, and you can search Raw Logs normally. Reports configured to run against the Logger also work as-is. And outside of the full USM Appliance integration, you also get the full Kibana interface with its visualization and reporting capabilities that have helped make the ELK stack so popular.

 

Speed

Most importantly, ElasticSearch is fast! Based on our testing on lab and production systems, we’re seeing searches return in seconds and large reports running in a minute or two. This makes your analysts more productive while making the overall USM Appliance platform more valuable for your security monitoring.

ElasticSearch Speed Comparison

The ELK Logger is more than just Raw Logs searches, the Castra Elastic solution is *fully* integrated, bringing its power to USM and appears to the system just like a normal Logger.

 

Expandability

Since it uses the ElasticSearch engine, this also opens up other possibilities including machine learning and anomaly detection using your log data. There are many other behavioral anomaly products out there, that can also sit on top of a Elastic data pool and provide new security insights for your environment.

 

Scalability

With Castra’s ElasticSearch you’re not limited by the amount of data you need to store. Need 4TB, 8TB, more? No problem, increase the storage size or add more nodes! Need redundancy? Also no problem, add more nodes! Elasticsearch was built to run as a cluster, so it can scale to dozens or even hundreds of TB of data.

 

Virtual Requirements

Total Cores: 8

RAM (GB): 32

Storage Capacity (TB) Compressed / Uncompressed: 10TB / 4TB

Virtual Interfaces: 2 x 1GbE

Virtualization Support: VMware ESXi 4.0+ Hyper-V v3.0+ (Windows Server 2008 SP2 and later)

 

3.0 Upgrade Details

Castra Managed Services has just completed version 3.0 of our ELK Logger for AlienVault USM Appliance.  This is a major update with many new features including:

  • Elasticsearch, Logstash, Kibana updated to version 6.8.2.  This includes the Role Based Access Control (RBAC) and SSL Transport security features from Elastic X-Pack.  We have also predefined admin and analyst roles for your team to use, but you can easily add more through the UI.
  • Data enrichment improvements:
    • Greatly improved NIDS data format with plain-text log view and hexdump-style PCAP view in binary_data.
    • Sensor and Context names now added to events.
    • Better OTX Pulse mappings in events (otx_pulse field).
    • Arrival time added to all events.
  • Updated index template for AlienVault event fields
  • Automatic nightly security updates (base OS only).
  • Improved data retention script.
  • New "elk" command for better CLI management and visibility.
  • Improved console and SSH login banners.  :)

There were also several minor updates, mostly under the hood, to improve the system:

  • New CLI tools: net-tools (arp), sysstat (vmstat/iostat/sar), htop
  • Historical data copy tools now included and copied to USM automatically.  These are primarily used after initial install to copy older AlienVault Logger data to the new ELK Logger, but can also be used for data reloading or emergencies.
  • Increased Logstash JVM heap for better throughput and buffering.
  • Native BSON and LXML libraries for faster data parsing.
  • Improved syslog and logrotate templates.

Finally, there were a few bug fixes to keep the system stable:

  • Critical fix to Logstash configuration to prevent logs with very old timestamps from being written into frozen indexes, which causes stalls and ultimately failures in Logstash.  Events with older dates are transformed so that they are not lost but instead written to the current day's index.
  • Fixed HeapDump output for Logstash - prevents Logstash errors from filling up the filesystem.
  • Code cleanup and improvements in core av-logger and fetchall components.
  • Security improvements in file ownership and permissions.

 

Want a Castra ELK Logger for FREE?

No problem! Contact us today to learn how!