Handling Invalid Events

All events arriving at CoolaData are marked as either valid or invalid. Invalid events are stored in a separate table, not included in queries.

Validity information can be seen in the Live Events page. The following three columns can be selected from the columns list:

  • validation: valid/invalid/pending (a new event that will be added to the schema if valid)
  • invalidComments: will specify the reason if the event is invalid.
  • extraComments: will specify any changes made to the event data.

Once you resolve the validity issues, newly received events will be valid. Previously received events remain invalid in the CoolaData database.


Reasons an event could be invalid and method of handling

  • Event Structure (JSON): If the event JSON is not a valid JSON structure the event will be invalid.
  • Missing Data: CoolaData trackers manage user_id and event_timestamp_epoch automatically. However, when sending events via REST API all properties must be included. Each event must be sent with the following 3 mandatory properties:
    • event_name
    • user_id
    • event_timestamp_epoch
  • Wrong Data Type: If the data received with an existing property is of a different data type, the property will be invalid. For example, an integer property sent with quotation marks (as a string) will be invalid. To update/add properties go to the Project – Properties page.
  • Property Quota: Each project is set up with a certain quota on the number of properties that can be used in the Data Scheme. If you reach the limit of the Data Scheme new properties will be invalid. To extend your Data Scheme quota, please contact your customer success manager.
  • Date Out of Valid Range: A time stamp in the future or in the far past will be considered an invalid time stamp and the entire event will be invalidated. The default values of future/past events are: past – 30 days, future – 1 hour. This setting can be configured for your specific project. Contact support@cooladata.com for further assistance.
  • Mapping Modes: A project can have one of two mapping modes:
    • In Locked mode, new events and properties are marked as invalid and will not be stored. If any of the properties in the event is invalid for any reason, the entire event will be marked invalid and will not be saved. To update/add events go to the Project – Events page.  To update/add properties go to the Project – Properties page.
    • In Self-Learned mode, new events and properties are automatically added to the project schema. Recognizing and registering new events and properties to the schema in Self-Learned mode can take up to 5 minutes, during which time new events will be marked as invalid.
      For more information on mapping modes see Project Configuration.
  • Event Size the size of the event sent to Cooladata can be no larger than 200KB.
    • Note: Events larger than 4.5MB will not be parsed and receive an error code as a response to the event sending request.


Querying Invalid Events

  1. Wait an hour or two after sending the events to CoolaData using a Tracker.
  2. Open a new CQL report (click the CQL button in the main menu).
  3. Select a date range in the top right hand corner (notice the invalids table is available for customer inquiring up to 7 days). To receive data on a longer period contact support@cooladata.com
  4. Paste in the following query:
    SELECT eventAsJson, reason, event_name, event_time_ts
    FROM invalids
    order by event_time_ts

    The eventAsJson property holds the original event sent to Cooladata, the reason property includes the reason for invalidating the event.

  5. To group events by failure reason, use the following query:
    SELECT reason, count(*) as invalids
    FROM invalids
    GROUP BY 1
Print Friendly, PDF & Email

Managing User IDs

There are times when the same user may be active on several devices (Mobile, PC, Tablet, etc). Being able to associate user activities, across different devices or applications, to the same user may reveal important insights, and have an affect on overall business performance.

Cooladata provides a mechanism to associate user activities from various devices to the same user identity. We will detail this mechanism below. You are welcome to contact support@cooladata.com with any additional questions you might have.

The Challenge

Consider the following business scenario:

  1. A user installs a game on their Mobile device.
  2. The user plays the game and registers.
  3. The user then installs the same game onto their Tablet.
  4. The user plays and logs in with their Facebook identity.

How many users would you count in this scenario? The worst case scenario is that your analytics platform treats the interactions of this single user as four different users:

  1. Interactions on Mobile pre-registration would be counted as one user (1).
  2. Interactions on Mobile post-registration would be counted as one user (2).
  3. Interactions on Tablet pre-login would be counted as another user (3).
  4. Interactions on Tablet post-login would be counted as another user (4).

A better strategy would be to track all these interactions as a single user.

How should you track this with Cooladata?

Any Event that is sent to Cooladata must provide the Property user_id. Cooladata expects you to use the user_id property to send a constant anonymous user ID, which is available for the application at any time.

In addition to that, with any event you can provide a user_alternative_id (Alias) as an Event Property. Cooladata expects you to provide the user_alternative_id when the user registered and you can gain access for a registered user ID.

Once an event is sent with a user_id and user_alternative_id together, Cooladata will associate all events from this registered user, based on user_alternative_id, to the pre-registered user, based on user_id.

Let’s review the five steps scenario as Cooladata expects you to track it:

Scenario Stepuser_iduser_alternative_id (Alias)uid
11234NACoolaData generated 'abc'
212349991234 and 999 events are mapped to 'abc'
36789NACoolaData generated 'xyz'
46789999Mapped to 'abc'

When you use one of CoolaData’s Trackers, CoolaData automatically enriches each event with a user_id that uniquely identifies the end user who performed the event. This user_id is saved as a CoolaData cookie on the device that generated the event and is uniquely associated with that user_id.

Sending your own user ID

In addition, you have the option to add your own user_id to each event. To assign your own user_id to an event add a property named user_id to the JSON sent to CoolaData.

Note – When you use the CoolaData REST API to send events, CoolaData does not automatically generate a user_id. In order to manage user IDs, you must add the user_id property to each event, or send it with the tracker init.

If you added your own user_id to an event then CoolaData stored the user_id that you sent as customer_user_id, the user_id that CoolaData automatically generated (as a cookie on the device) as internal_user_id. Both types are treated as the same user and maintain a one-to-one relationship between them.


Consolidating multiple users

In addition to the above, you may want CoolaData to manage multiple users as a single entity. This may be useful when you want CoolaData to consolidate the events of users that log in anonymously (or with a different login name) on the same computer or users that log in using the same name on a different computer.

To consolidate multiple users:

  • Add an alternative_user_id property containing your website/app login name to an event along with a user_id property. This will map multiple customer user IDs to the same internal_user_id.
  • The two IDs only need to be sent once to make the connection. Sending them again will have no impact.
  • Sending the 2 user identities for coupling can be done either by sending them together in an event or by using the user identities matching API. 


  • To display the user_id that you sent to CoolaData, query the customer_user_id property.
  • For better performance, in CQL queries, use user_id whenever possible instead of the customer_user_id or alternative_user_id.


User management at Cooladata

Cooladata’s data model manages three user_id fields: customer_user_id, user_id and user_alternative_id.

  • customer_user_id is a mandatory property in any event that is being sent to Cooladata. Use this Property to send the anonymous ID, which is available at all times. In most cases this is an ID that is saved in a cookie called cd_user_id.

  • user_alternative_id (Alias) is an additional user property. You can send it with any event you choose and it’s not mandatory. This Property should be used when your user identifies themselves uniquely for the first time (e.g. via logging-in), and you can send it when available. It should be sent together with the user_id you send with any event.

  • user_id is managed by Cooladata and is used to keep a coherent user identity. The user management module of the Cooladata platform generates the user_id to keep track of the various interactions and map all interactions to a global user ID.

    Important note: the user matching mechanism (user_alternative_id) is currently in beta. to use it, contact support@cooladata.com – This feature will not be active prior to contacting us.

Other scenarios that can impact counts:

IOS – iTunes & App Stores – Automatic Downloads

Apple may install an app of multiple device from a single install event. Users install the app on their mobile device and the user iOS configuration (Settings -> iTunes and App Store) may enable Apple to automatically install the same app on additional devices such as tablets or mac owned by the same user.

iOS settings

Many customers send an install event from their mobile measurement provider (such as AppsFlyer or Kochava). In this case, you will see activity from devices that never had an install event.

If you are using IDfV (ID for Vendor) for the individual user, you will also see two users, where it is the same user with different devices.

As a result you may prefer generating an id per user per app and send the device as a property instead of using the IDfv.

This may also help when users continue a session between different devices using IOS8 Handoff.

How User IDs are Assigned in CoolaData:

CoolaData assigns a user_id so that it will persist across sessions. If applicable, we also persist the user ID across apps and upon installs and uninstalls. The following text details how user ID assigned in various platforms.


The CoolaData JavaScript SDK automatically generates a unique user ID. The user ID is generated the first time the library is called, and stored in a cookie for future usage. The cookie is stored by default for one year, and applicable for the toplevel domain.


If the app is using the AdSupport.framework, we’ll use the Advertising Identifier.
For apps that do not have the AdSupport.framework included, CoolaData will default to using the identifierForVendor as the user_id.


If the app is using Google Play services 4.0+, then we’ll use AdvertisingIdClient.
If the app working with the READ_PHONE_STATE permission, then we get Secure.ANDROID_ID; otherwise, use UUID.randomUUID().


User Time to Expire

The data you tracked is saved in Cooladata indefinitely.

However, user ID associations are kept for up to 90 days. Users that have not been active for more than 90 days will be treated as new users. This means that they will not be associated with any data previously provided about them. Note that users can be assigned multiple IDs, and logging in using any of the ID’s associated with them will keep them active in Cooladata for an additional 90 days. To update user’s data after more than 90 days, send the data again with one of the first events once they’ve logged in to the system.

Print Friendly, PDF & Email

Managing Sessions

What is a session?

In CoolaData, a session starts when a user accesses your website/app and ends after that user has not performed any event for a specified amount of time.


Sessions table

CoolaData maintains an aggregated table of session information called the Session Table. This is the source from which CoolaData extracts behavioral analytics, meaning analysis and reporting of the sequence of user events. For example, in Funnel and Cohort reports. Some reports extract data from this Session Table instead of from the Events Table. Extraction from the Session Table is faster than from the Events Table. However, the Session Table does not maintain information about the properties of an event.


Session ID

CoolaData automatically assigns a unique identifier (Session ID) to each session.

You can specify your own session ID by including the session_id property in the event sent to CoolaData. If you send your own session ID, then this information is stored as a property called customer_session_ID. CoolaData still maintains its own CoolaData session ID in addition. You can then query using either customer_session_ID or CoolaData’s session_ID.

Note: Properties that are calculated per session are based on CoolaData’s session ID (session_id), and not on your session ID (customer_session_ID).


Session properties

Some CoolaData properties are collected per session:

  • session_duration – The duration of the session, in milliseconds.
  • session_path – A session path is a sequence of event names (funnel) in a session. CoolaData provides various visualization options showing the path of events in a user session, such as a Funnel report, Cohort report or Path report.
  • session_id – A unique ID assigned to a specific user for the duration of that user’s visit (session).


Ending a Session

CoolaData automatically stores the events of a session when it ends. Therefore, it is important to ensure that a session ends.

CoolaData automatically ends a session after no events have been sent for that user for a specified amount of time – by default, 30 minutes. This can be changed from the Project – Settings page.

If a session does not end within six hours, then CoolaData automatically ends it and stores its data. Therefore, if a session is still getting events, then it may take a while until you are able to query that data.

Best Practices:

  • For a website session, we recommend a timeout of 30 minutes.
  • For an app session, we recommend a timeout of 5 minutes.
  • If your website has videos that include advertisements: if ads are sent as events we recommend using the 30-minute timeout. However, if the ads are not sent as events, we recommend you lengthen the session timeout to longer than the video, which might run more than an hour before sending an event.


Include in Path

Each event that you define in CoolaData has an Include in Path setting that specifies whether an event is shown in the CoolaData behavior analytics visualizations (Funnel, Cohort and Path Analysis reports) or not. The Include in Path field should be off for certain types of repetitive events that are not needed for a session analysis.

For example, a keep_alive event that is generated when the device periodically verifies that it is still connected. Because keep_alive and page_load events  are sent repeatedly, the session will only end after six hours unless you turn the Include in Path option off because CoolaData will continue to leave the session active for 30 minutes (default) after each event.

Regardless of the value selected in the Include in Path field (meaning even if it is off), the event still affects the session_duration (which is the duration of the session, in milliseconds) and is still counted in the session_event_counter (which is the number of events in the session).

Print Friendly, PDF & Email


How can I retrieve event_time_ts in different formats?

Event_time_ts shows only minutes because that is the format our UI supports. However, you can drill down to the seconds and milliseconds by using CQL functions. For instance:

  • time(event_time_ts) returns time of the event in HH:MM:SS format
  • string(event_time_ts) returns time of the event in HH:MM:SS.ms format
  • timestamp_to_msec(event_time_ts) returns event time in milliseconds

What is the default timestamp format?

%Y-%m-%d %H:%M:%S

How do I query a specific range of hours within a single day?


How do I query a 24 hours data, referring to my local time?

Cooladata timestamp is based on UTC standard. So, you first need to check for time differences.
For instance, if Israel is UTC +3, and you wish to query the entire 24 hours of a specific date (for instance July 8th, 2016), the query should look as follows:

select hour(date_add(event_time_ts, 3, 'hour')) as israel_hour, count(*) as cnt
from Cooladata
where date_range(between 2016-07-07 and 2016-07-09) and
date_add(event_time_ts, 3, 'hour') >= timestamp('2016-07-08 00:00:00')
and date_add(event_time_ts, 3, 'hour') < timestamp('2016-07-09 00:00:00')
and filters(context)
group by 1
order by 1

Does Last 7 days include today?

The last N days includes all the last N days up until yesterday, not including today.

Is there a limit of rows for exporting CSV file in Cooladata app?

Yes. there is a limit of 20k rows for exporting via Coola app.

What does the ‘Can Edit’ permission enable me to do on a dashboard?

Sharing a dashboard with Edit permissions, it enables you to add or remove reports from the dashboard, reposition them, but it does not give you the ability to edit the reports themselves.
When someone else creates a report – you cannot edit it unless you are the project’s Admin.
This is because these reports can be used in other dashboards or publications that you might not have access to.

Is there a query quota?

Cooaldata limits the maximum rate of incoming requests and enforces appropriate quotas. Specific policies vary depending on resource availability, user profile, service usage history, and other factors, and are subject to change without notice.

The following limits apply to  query-type function calls.

  • Concurrent rate limit: 1 TB of concurrent queries + 1 additional query of unlimited size, up to 20 concurrent queries.
  • Daily limit: 20,000 queries.
  • Maximum query length: 256 KB1
  • Maximum response size: 128 MB compressed2

1Approximate size based on compression ratios.
2Sizes vary depending on compression ratios for the data; the actual response size may be significantly larger than 128 MB.

I got this error: “External data source format is invalid” – what should I do?

This is likely to occur when querying an external database you manage, and the firewall settings are not configured to allow Cooladata’s IP addresses.
These are the two IP addresses you need to allow access to:

  • 52.44.248

Print Friendly, PDF & Email

Anonymizing Personal Data

AS part of the GDPR efforts, Cooladata now enables you to anonymize personal data in your project.  Since you control what data you send to Cooladata and what is stored, you are responsible for defining what is personal information and what is not.

This feature will allow you to hash properties marked as personal user information.  

Notice that we automatically collect IPs so if you need the IPs we have collected to be hashed before they are stored you are responsible to use this functionality to set this up.  Hashing IP’s will not affect the geolocation enrichment we provide out-of-the box.

Keep in mind that this feature will only hash future data being sent to Cooladata and will not update historical data.

Marking Properties as Personal Information

  1. In Cooladata, go to the project menu -> properties – to see the project’s properties list
  2. Search for the wanted property and click on it
  3. In the Edit Property side bar mark the “anonymize data” to mark this property as data that should be hashed:

Notice that STRING type properties will be hashed using MD5 so the data will remain unique per value, but INTEGER,FLOAT and TIMESTAMP data types will be Nullified.


Setting up a project wide condition for anonymizing data

You can also set up a project-wide condition to define that only data that meets this condition will be hashed. Due to legal issues, this is done offline using a form. Please contact support@cooladata.com or your CSM to set this up.

Once you set up a condition, all properties with “anonymize data” checked will be hashed once the condition will be true. Even if the property will be marked after setting up the condition.


Print Friendly, PDF & Email