CX Journeys Fall 2020 Release Notes

Released starting 3 September 2020

Path Builder

Path Builder now has these enhancements

Expand events (Events breakdown by properties)

Users can now choose to expand the results of path analysis by a particular property to see how it may impact customer journeys. Each event can be expanded by a single property. For example, users can break down the “add_to_cart” event by the “item_name” property to see the most common items added to cart.

To to do so, under path analysis click the “Expand Events” button, and choose the desired event:

Reverse order

Users can create Sankeys that appear in reverse order to focus on the end of the path and present the results from right to left. To enable reverse order, on the report settings turn on the “Reverse order” toggle: 

Hide drop offs

For some of the journeys use cases, the focus is more on the journey’s side and less on the drop-offs. Now, with a simple click (under visualization settings) users can hide drop offs from the Path report visualizations. 

Print Friendly, PDF & Email

Table Partitioning by Column

Cooladata’s events and sessions tables are automatically partitioned by time.  Cooladata also enables the partitioning of other external tables in your project by a timestamp or date column.

Partitioning makes sure you run only on the selected time range in the table instead of the entire table, improving performance and minimizing query run time. Partitioned tables can be created by models, aggregation tables , Google Cloud Storage or Gmail integrations and are only available for querying in Standard SQL dialect.

Table Partitioning Settings

Table partitioning requires two settings:

Partition Column

Defines the column in the table which the table is partitioned by. Supports DATE or TIMESTAMP data type.

Data written to a partitioned table is automatically directed to the appropriate partition based on the date value (expressed in UTC) in the partitioning column.

When querying the partitioned table, a filter on this partition column will automatically direct the query to run only on the relevant partitions.

Require Partition Filter

Defines whether queries running on this table will require the partition column in the WHERE clause or not. When TRUE, queries selecting from this table without the partition column filter will fail.

Creating a partitioned Table using an Aggregation Table

Aggregation Tables automatically run scheduled calculations and data aggregations and save them in a separate, permanent table in your Cooladata project. To make this table partitioned, the Aggregation Table query should be in Standard SQL dialect (creating a partitioned table using Legacy SQL is not supported), and the following fields need to be filled in before the aggregation table’s first run:

 

Notice that after the aggregation table’s first run the table can only be queried using Standard SQL dialect.

 

Creating a partitioned Table using a Model

Models are designed to add R and Python capabilities to your Cooladata’s workflow. The results of the R or Python script is saved to a table in your Cooladata project.
To make this table partitioned, you need to fill in the following fields before the model’s first run:

 

Notice that after the model’s first run the model’s table can only be queried using Standard SQL dialect.

 

Creating a partitioned Table using an Integration

Gmail or Google Cloud Storage integrations allow you to upload an external file to a table in your Cooladata project.
To make this table partitioned, open the advanced section in the integration settings and  fill in the following fields before the integration’s first run:

 

 

 

 

 

Notice that after the integration’s first run the table can only be queried using Standard SQL dialect.

Print Friendly, PDF & Email

Sending Data

Is there any size limit for each event I send?

Event size should not exceed 100KB.

I’ve noticed events on ‘Live Events’ view but cannot see them when I query.

There are two reasons that this is likely to happen:

  1. The events haven’t been loaded to the data warehouse yet (Google BigQuery), and thus cannot be queried. In this case, wait for about an hour until the data loading is completed
  2. Some of the events’ properties were not sent in the correct format (either wrong type, missing etc.) and were consequently sent as invalids. You can query all invalids up to the previous 7 days, using the following query:
    select * from invalids
    where date_range(last 7 days)

Why am I getting the following error, when I am sure this property exists in my project: “Field ‘session_duration’ not found in project, Coolalog no:5522517”?

This error occurs when trying to pull data from two different partitioned tables: events table, and session table. To solve it, join both tables using mutual property appears in both, e.g. user_id.
For instance, the following query will produce this error as event_name is a user scope property (saved in events tables), whilst session_duration is a session property (saved in sessions tables):

select event_name, session_duration
from Cooladata
where date_range (last 7 days)

Does Cooladata support multiple customer identities?

A user can start as anonymous user (hash key generated automatically), and then become a registered user. We support one old identity per user.
Once the user is sending both identities within the same event, we know how to convert it to the new identity.

Is session_id mandatory?

No

Which columns are automatically generated?

based on session_ip we are generating ip_country, ip_region, ip_city, ip_longtitue, ip_latitude.
based on DUA (device user agent) we are generating brand and model.
based on timestamp, we are generating multiple columns (hour, day, month, year, week…)

See our common properties documentation for more details.

Is the API Token the same as the App Key for using SDK implementation?

The AppKey is used to send events. It is the same regardless of how you send the events (REST, JavaScript etc.). However, the Query API is your user token and is used for querying the system. This can be retrieved by logging into Cooladata and clicking on the avatar in the top right-hand side of the screen.

How can I differentiate test data and organic data sent from my users?

There are several ways of differentiating between test data and real data:
A different project
A property
Both above methods require code intervention (in your app) to distinguish between real and test data.
In addition, if you have a distinct (and not too large) set of either devices or users that are generating test data, you can build segments that reflect those “test users”/”test devices” and filter them out in the dashboard slicers.

Handling Personally Identifiable Information (PII)

Cooladata takes the utmost precautions to ensure the security of your data in the cloud and continually upgrades with the latest security options.

Cooladata accepts any event properties that you send without filtering them. However, even so, we advise you not to send sensitive personal information (such as credit card numbers) that may help a malicious entity identify someone.

Here are a few tips for protecting personal information:

  • Conceal personally identifiable information. For example, by scrambling, cloaking, encrypting, faking or hashing it.
  • Send a person’s location, instead of their IP address.
  • Send only partial information, such as a person’s country instead of their IP address.
  • Do not send combinations of information that may help someone piece together who the person is, such as session IP, address, gender and age.
  • Read here for the feature we have released in order to help our customers comply with GDPR and other PII regulations.
Print Friendly, PDF & Email

Common reasons for data discrepancies

Companies often use several tools to understand their user’s behaviour. Alongside Cooladata, most companies also use Google Analytics or their own DB for comparison. 

We suggest investigating discrepancies if there is more than a 5% difference between Cooladata and other tools. Any less is likely not material enough to warrant a full tracking audit, since often analytics are used to identify trends (e.g. how fast are we growing?), rather than exact numbers. If the difference is greater than 5% across all events, or specific events don’t match between systems, then further investigation is called for.

Check this guide to identify common reasons for data discrepancies between various systems:

Timezone

Cooladata’s default timezone is UTC. When comparing data between Cooladata and other system such as Google Analytics, consider the time zone differences.

Double-check your query

When weird numbers appear in a report, it’s not always a data issue. Sometimes the report is just not querying what we intend it to query. Make sure you are querying the correct date range, and no filters are applied, filtering out relevant data.

Invalids

Cooladata automatically validates the data sent to Cooladata in order to prevent Garbage In – Garbage Out situations. Events marked as invalid are not stored with the rest of the valid events, but in a separate, designated table for invalid events, which you can query to check what went wrong. See Handling Invalid Events to learn more.

Data Sampling

Google Analytics sometimes use sampled data in reports, causing discrepancies in the numbers from Cooladata. Google Analytics sampling occurs automatically when more than 500K sessions are collected for a report. Google Analytics state that a report is based on sampling in text above the report. When comparing Cooladata to a sampled Google Analytics report, discrepancies are expectable.

Session Definition

A session in Cooladata starts when someone visits your site or app, sending an event, and ends after thirty minutes of inactivity. The session duration is calculated as the difference between the first and last event in that session. This thirty minutes timeframe is a configurable parameter. If you are the project’s admin, you can see this parameter under ‘Session timeout“ in your project settings page. Most analytics tools, such as Google Analytics use this thirty minute definition, which might cause discrepancies when comparing session duration or number of sessions if you set this parameter to be different than thirty minutes in Cooladata. Also, consider that Google Analytics will count additional sessions for clicks on AdWords campaigns, and will hard stop all sessions at midnight, whereas in Cooladata sessions occurring across midnight (starting before midnight and ending afterwards) would be stored as one session. Other mobile analytics tools platforms also end sessions if the user moved the app to the background for more than a minute.

Events are sent differently

A common cause for discrepancy is the way the events are sent to Cooladata and other tools. For instance, if one tool is receiving events from the server-side and the other from the client side, differences in numbers will most likely occur.

Even if both tools are sent from the client side, the code needs to be checked. Sometimes, there is a logical condition for sending an event to one tool which is not the same as the code sending the event to the other tool. When using JS SDK, the location of the trackEvent code is important. If the call sending the event to one tool is at the top of the code and the call sending the event to another tool is at the bottom of the code, there might be some discrepancies, due to an error in the code or if the user manually closed the window before the trackEvent function was called.

Bots and Test Users

Some tools automatically filter out events created by bots, Cooladata does not. Cooladata does have several solutions available if you wish to slice out bad IP’s or bots. To find the best solution for you, contact your Customer Support Manager.   

If your operational DB automatically cleans test user’s activity, make sure you filter out test users in Cooladata as well.

Funnel and Conversion Definitions

Funnels in Cooladata count distinct (unique) users who completed the funnel in the date range in question, in the time window set in the report. Conversions are sometimes defined differently in other tools. For instance, Google Analytics count the number of sessions in which the funnel’s steps were completed.  

Notice that if you choose to set the funnel in Cooladata to show users who completed the funnel by X days, the funnel will only include users who did the first event X days before the end of the report date range when looking at the last week or month. This is done in order to give users who came in at the beginning of the date range, the same “chance” to complete the funnel as users who came in at the end of the date range.

Redirects and Self Referrals

When looking at Cooladata’s “referring_url” and “referring_domain” you should see the url and domain the user was referred from. Sometimes, you see url’s and domain’s that you do not expect, such as your own url. This happens when your site uses redirecting rules, usually set up by your site admin.

Cooladata Sessions Table

Cooladata stores your data in two separate tables: one is your event table, in which every row is an event, and the other is your sessions table, in which every row is a session. The event table holds all the event-level data and event-scope properties, as well as user and session scope properties. The sessions table holds session-specific properties (such as session duration and the session path) as well as session and user scope properties. In order to optimize performance, Cooladata automatically shifts your queries to run on top of the sessions table if all the data you are searching for is there. For instance, Daily Active Users (DAU), counting number of unique users per day will run over the sessions table, but if you want to count Daily Active Payers (pDAU), you will have to run over the events table in order to add a filter for the payment event. Shifting between the sessions and events table might cause for slight differences due to the fact that when running over the sessions table, the date range is filtered according to the session start time (session_start_time_ts) whereas when running over the events table the date range is filtered according to the event (event_time_ts)  timestamp.

Print Friendly, PDF & Email

Opting out users

*Please note that this documentation is for a BETA feature. Please contact support@cooladata.com or your CSM for more information.

The opt-out features allows you to mark a user as an “opted-out” user. This will block any future events of this user from being processed or stored.

The GDPR requires you, as a data controller to inform your users (“data subjects”) with a clear explanation of how, where and by whom their data will be processed in accordance with the requirements of Articles 13 and 14 of the GDPR. Controller are also required to receive their consent for the collection and processing of any data they will share through your services unless the controller can rely on another legal basis (the list of permitted legal bases is described in Articles 6 and 9 of the GDPR). This question, or request for consent, is your responsibility as a data controller to invoke, and we imagine will look different across your different websites or your mobile apps. We encourage our customers to carefully assess which legal basis they rely on for each processing operation and display the relevant information required by the GDPR. If you receive an objection from your user, this feature should be used in order to stop tracking its activities.

Opting out a user will be done either by the SDK or a rest API request to Cooladata, by sending an event saying that the user has opted out of data collection. Our system will flag that user and will block all future events or sessions of that user. The opt-out operation will be carried out using customer_user_id. Since Cooladata allows you to link between multiple identities of the same user,  opting out will also work for all the alternative identities of a single user, and not just the records received with the user id that asked for erasure or opting out. Notice that opting-out will only block events sent to Cooladata, but uploading user data from external data source or integrations will not be affected, and it is your responsibility to ensure deletion of this user from these data updates. 

Opt-out event

The structure of the opt-out event will be like so:

{“event_name”:”cooladata_opt_out”,”user_id”:”<user_id>”}

This event can be sent from either the SDK by calling the track_event function (see the SDK documentation) or by sending a REST API call like so:

https://api.cooladata.com/v3/[app_key]/track

Content-Type: application/x-www-form-urlencoded

Payload: 
{events:
     [
      {"event_name":"cooladata_opt_out",
        "user_id":"<user_id>
     }
    ]
}

 

Investigating the Opt-out event

The actual opt-out event will be stored in cooladata, but the events following the opt-out event will not be stored. This means that sessions will be cut off and the “opt-out” event will be the last event stored in those sessions.
You can run the following query to investigate users that requested to be opt-out in the last 7 days:
select customer_user_id, event_time_ts as time_of_request
from cooladata 
where date_range (last 7 days) and event_name in ("cooladata_opt_out")

 

 

Print Friendly, PDF & Email