Technology features

 
  • complex and quality output data - Clipen provides complex and quality atomic clickstream data on hit and session granularity levels (see also Data characteristics); corrupted or improper data are rejected throughout the data processing using data quality management, while always achieving output data consistency and accuracy
  • near real-time data processing - Clipen incrementally produces sessionized data in near real-time; when the session data processing is completed after taking a processing latency and the session is recognized as finished by expiration of a session timeout, only one more essential latency is needed for the session data post-processing in order to finally provide the data for extraction (the measure of the latencies relies on actual performance and burden)
  • high performance and scalability - the architecture of the Clipen engine is built on the concept of data processing scalability that allows the technology to be deployed to high-volume websites as well as enabling growth together with growing needs
  • support for SMP and clustering - Clipen takes advantage of machines with multiple processors (including multi-core variants) and thus enables an increase in the overall performance; clustering is another cost-effective way to increase Clipen performance using entry-level servers and engaging them in a cluster when their additional performance is needed (this is a cost-effective incremental building of the hardware infrastructure)
  • support for multiple websites1 - Clipen is designed to process clickstream data from many independent websites simultaneously (a website may be treated as independent if it utilizes autonomous session tracking)
  • processing customization using Perl - clickstream data processing within the Clipen engine may be customized using custom routines written in the widely used Perl language; the customization is allowed for several isolated parts of the processing in order not to affect the general process but to achieve the required processing modification where possible
  • HTTPS support - the Clipen engine is capable of processing secure HTTPS (i.e. HTTP over SSL/TLS) communication thus enabling the creation of a complete picture of website users' browsing experience as well as supplying data on secure commercial transactions
  • non-intrusive network sniffing with aggregation - passive network sniffing is an entirely non-intrusive clickstream data collection approach that creates no additional load to the network segment where the tracked website is sniffed; the communication of all tracked websites' HTTP servers available on-line can be sniffed simultaneously, and unwanted clickstream data may be easily filtered out (for example internal traffic on the tracked website)
  • privacy protection - Clipen engine allows to mask values of any private or confidential data (for example passwords or credit card numbers) that may be present within sniffed network communication; original values of such data are not stored in the Clipen database then
  • RDBMS storage with SQL support - processed data are stored in a relational database whose management system (RDBMS) provides a couple of standardized SQL interfaces including native, ODBC, JDBC or DBI connectivity; the Clipen database has a simple format optimized for ETL and data warehousing
  • external control of data extraction - the opening and closing of the extraction window is fully driven by the Clipen user's external ETL application which is extracting processed output data from the Clipen database. The data may be extracted only within the open extraction window; when the extraction window is closed after data extraction, the processed data available within the last window are automatically deleted from the database and no longer available
  • configuration through web interface - the processing of the tracked website is configured by the ClipConf configuration tool that uses for its GUI the common environment of a web browser; this feature allows the tracked websites to be configured from almost any computer connected to the Internet
  • management console - Clipen management is provided by an interactive management console that can be remotely accessed using common operating system means for accessing from a remote terminal (for example the ssh secure shell)
  • engine diagnostics - the Clipen engine provides powerful diagnostic tools in order to allow performance optimization and tuning of the technology; the diagnostics is available through the ClipConf tool
  • automatic recovery - each part of the Clipen engine (except RDBMS) has the capacity for automatic recovery if an unexpected failure occurs; Clipen preserves data consistency if a failure is not a fatal error and the Clipen database with RDBMS is not affected - data available for extraction from the Clipen database are always consistent if the database file is not corrupted
  • error email notification - Clipen may send an email notification to the Clipen administrator in order to indicate a threat or an occurrence of the most critical errors (for example hardware errors like low hard disk space or a broken network line for sniffing); this feature may prevent failure or shorten recovery time
  • automatic update - Clipen automatically updates itself from a remote server; alternatively, the updating may be controlled manually or may be disabled entirely (however the last variant is not recommended)
  • low-maintenance requirements - Clipen is a stand-alone engine that does not require human supervision, although like any production technology its operation should be remotely monitored using its diagnostics tools
  • easy deployment - the technology is completely non-invasive for the tracked website and does not require any website changes to enable the clickstream data processing; just a connection to network line by appropriate network element (usually through a port mirroring of a network switch) and simple configuration of Clipen including tracked websites settings