As a member of the ExtraHop Networks Data Science Research & Development team, I was responsible for implementing a content delivery network (CDN) to move data from the company's data centers located around the world to a central location to facilitate Machine Learning powered cybersecurity detections. ExtraHop's Reveal(x) 360 platform provides real-time visibility and analytics for networks, and the consolidation of data from the various data centers was necessary for the platform to function effectively.
To implement the CDN, I utilized a range of technologies, including AWS Kinesis Data Streams, serverless SQS queues, and S3. These technologies allowed me to efficiently and securely move large amounts of semi-structured data across the ExtraHop network of data centers.
ExtraHop Cloud high-level architecture [source: ExtraHop]
The use of AWS Kinesis Data Streams enabled me to process and transfer the data in real-time, providing immediate visibility and analytics to the Reveal(x) 360 platform. The serverless SQS queues provided a scalable and reliable way to store and transfer the data, while S3 ensured that the data was securely and durably stored.
The service uses a combination of supervised and unsupervised machine learning algorithms to analyze data in real-time and provide alerts when suspicious activity is detected. The algorithms used by ExtraHop are part of an ensemble, which means they work together to provide more accurate results than any single algorithm could produce on its own. ExtraHop's service is designed to be simple to use, requiring minimal setup and configuration, and can be easily integrated into an organization's existing security infrastructure.
The CDN that I implemented resulted in a significant improvement in terms of GB/$ space, with a 400x improvement compared to previous methods of data transfer. This allowed ExtraHop to move large amounts of data across its network of data centers without incurring excessive costs.
Overall, the implementation of the CDN was a successful project that allowed ExtraHop to consolidate data from its network of data centers and improve the performance and functionality of the Reveal(x) 360 platform.