Models – New!

Models are designed to add R and Python capabilities to your Cooladata’s workflow.

From the models page, you can create tasks based on R or Python scripts. This opens up a new dimension of exploration and analysis of your data, encompassing all capabilities of these languages.

Models are similar to Aggregation Tables:

  • They can run on any data source in you project, including events, tables, and linked data sources.
  • They can be scheduled to run automatically on a set frequency.
  • The model generates results that are stored in a table, which can then be queried from anywhere, including reports, publications and Query API.

 

Creating/Editing a Model

  1. From the top menu, click Models.
  2. A list of your saved Models is displayed, or if you haven’t created any yet, an Add Model button is shown.
  3. To create a new Model, click the + button at the top right or the Add Model button, if your list is empty. This opens the Models gallery:
  4. From here, choose the type of builder you wish to create. The various builder types are explained below.
  5. Click any task (row) in the Models list to open it for editing, or click the options icon at the right end of the row to see the task history, run it, delete it, or see related Jobs.

 

Model Types

Cooladata provides builders for custom or predefined models. Custom model builders allow you to use any query and script you wish.
Predefined models (coming soon) will allow you to apply advanced machine learning algorithms to your data with just a few clicks and without writing any script.

 

Custom Models – R Script, Python Script

In the new Model gallery, choose either R or Python script to start writing your own custom models.

  • In the Model editor page, enter the following details:
    • Model Name: The name of the Model task. Supports any text. Enter something descriptive to help you identify the task.
    • Data (CQL): enter any query, on any data source, including all tables and linked data sources in your project, that will serve as input for your script.
      • The results from the query above are saved to “data”
      • Notes:
        • Do not use the “filters (context)” or “date_range (context)” features in queries run via Models. This is because there is no (report/dashboard) context for these kind of queries, and therefore the query will fail.  Instead write explicit date range and conditions.
          Tip: use the report options “Show CQL” to see the final query (without “context”) when copying a query from a report to a Model.
        • Does not support “select * from cooladata
    • Script – R: enter your script here. We recommend running your script in a dedicated R IDE (preferably on Unix) and debugging it there before using it in Cooladata.
      • Some libraries are pre-installed in your project and can be used. You can also add other libraries to run in this specific script from the “Additional libraries” field below the script.  The libraries pre-installed in the R script are: ggplot2, plyr, reshape2, RColorBrewer, scales,grid, wesanderson, RJDBC, devtools, corrplot, testthat
      • The results of the script must be in a table format. Save your result to coolaResult, like so:
      • coolaResult <- data
      • This will ensure that the data is stored in a table to your project (see Table name in settings).
      • Cooladata uses S4 OO when running your R script and currently supports R version 3.4.3 and any earlier versions.
    • Script – Python: enter your script here. We recommend running your script in a dedicated Python IDE (preferably on Unix) and debugging it there before using it in Cooladata.
      • Some libraries are pre-installed in your project and can be used. You can also add other libraries to run in this specific script from the “Additional libraries” field below the script.  The libraries pre-installed in the Python script are: JayDeBeApi, Pillow,h5py, ipykernel ,jupyter, matplotlib, numpy, pandas, scipy, sklearn
      • The results of the script must be in pandas data frame format. Save your result to coolaResult, like so:
      • coolaResult = data
      • This will ensure that the data is stored in a table to your project (see Table name in settings).
      • Keep in mind that currently Cooladata supports Python 2.7.12
    • Additional Libraries: If you need additional libraries, enter them in the Additional Libraries section by typing the library name and press enter). For R, you may enter any Library available in CRAN repository. Keep in mind that this ensures the library is installed but you still need to import it in your script (you may use aliases if you wish). 
    • Run: click ‘Save and Run’ to save the Model and execute the query and script.
      • Run time will depend on the complexity of the query and script. You can either close the page and return to it later, or keep it open and the results will be shown once the first run is complete.
      • The default preview shows you the top 50 rows of the table created. You can also examine the run logs from the display.
      • You can see the full table by querying it from a CQL report using:
        select *
        from table_name
    • View Logs: The “view logs” section is available after first run. It shows logs for the 3 steps of the Model Run:
      • Step 1: Initialization: this will show errors when installing invalid libraries or if there was an issue creating the environment to run your script.
      • Step 2: Running the script, this will show warnings and errors in the R/Python script code or in your CQL query that retrieves the input data.
      • Step 3: Saving the data to the DB, this will show errors when you have issues with invalid column names or table schema.
  • Settings:
    • Table name: the table to which the data will be saved. Table names are case sensitive, and cannot include spaces or special characters. Take care when using an existing table, as existing data might be overwritten.
    • Write mode:
      • Append: new rows are added to the table each time the query is computed.
      • Append and update: new rows are added to the table each time the query is computed, and existing rows that match the unique key you selected will be replaced with the updated data.
      • Replace: The entire table data is overwritten each time the query is computed.
    • Notify on Failure to: email/s to be notified if any scheduled run fail.
  • Schedule: when to run the query.
    • Active: turn this off to prevent any run of this task. Note that this will also block Jobs from running it.
    • Frequency:
      • Daily: At a specific hour of the day (UTC).
      • Weekly: On a specific day of the week, at a specific hour of the day (UTC).
      • Monthly: On a specific day of the month, at a specific hour of the day (UTC).
      • CRON: Set the frequency by specifying a CRON expression. The CRON expression should be in the format of – minutes (0 – 59) hour (0 – 23) day of month (1 – 31) month (1 – 12) day of week (0 – 6). The star symbol (*) should be placed as a wild card. For example, this CRON expression will run the Aggregation Table daily at 3:30 am UTC: 30 3 * * *. You may refer to www.cronmaker.com for a description of CRON expression syntax.

 

Technical Details

The models are deployed using Dockers with a static IP so you can use them to access IP white listed services. The IP address is 18.205.8.38
Furthermore, notice that the script written in the model will be running on a Linux environment.

 

Querying Models

To query a Model, in a CQL query, as with any table in your project, state the table name in the FROM clause, instead of “cooladata”. For example:

FROM users_table

You can also use these tables as the data source for various report builders, such as KPI. See the report documentation for more information.

To use a date picker on a query from a table, use the TABLE_DATE_RANGE function – see Date and Time Functions for more information.

Models saved to your project can be found in the Schema, under Models.

  • Expand any table to see the table’s columns and their data types.
  • Drag&drop a table/column name to use it in a query.

 

 

Print Friendly, PDF & Email

Publications

Publications enable you to send an email containing a set of reports to a list of recipients, either registered to CoolaData or not.
A publication email displays the report charts, and includes a CSV attachment of the data of each report.
Any user can send a one-time publication from a report page.
Project admins can also create and manage scheduled publications.

Publications can also be sent as white-label, with your branding and mailing addresses. Contact your Customer Success Manager, or email us at support@cooaldata.com for more information.

 

Sending a One-Time Publication

  1. From any saved report page, click Options and select Publish. The following displays:
    3-28
  2. Define your publication:
    • Title: the title shown in the email.
    • Reports to Publish: the reports to include in the email.
    • Recipients List: any (valid) email address, whether registered to Cooladata or not. Add yourself as a recipient to receive a copy of the publication email.
  3. Click Publish now to send the publication. The email should arrive within a few minutes.

 

Defining a Scheduled Publication

You can create a new scheduled publication either from a report, as explained above, or from the publications page.

From Report

  1. From any saved report page, click Options and select Publish. The following displays:
    3-28
  2. Define your publication:
    • Title: the title shown in the email.
    • Reports to Publish: the reports to include in the email.
    • Recipients List: any (valid) email address, whether registered to Cooladata or not. Add yourself as a recipient to receive a copy of the publication email.
    • Frequency: the schedule for sending this publication email:
      • Daily: the email will be sent every day, at the hour you select (UTC).
      • Weekly: the email will be sent on day of the week and at the hour you select (UTC).
      • Monthly: the email will be sent on the day of the month and at the hour you select (UTC).
      • CRON Expression: define any schedule valid by CRON expressions. See www.cronmaker.com for a description of CRON expression syntax.
  3. Click Save to save the publication. It will be sent on the schedule you defined.
  4. You can also click Publish now (before or after saving the publication) to send the email right away.

 

From Publications

  1. From the Project menu (cogwheel icon at the top right), and select Publications.
  2. Here you can create new publication, and manage your publications.
  3. Click the + button at the top right to create a new publication.
  4. Define your publication, same as From Report (see above).

Note: A project can contain up to 5 saved publications. To enable your project to send more publication, contact your Customer Success Manager or email us at support@cooaldata.com.

Print Friendly, PDF & Email

Jobs

Jobs allows you to schedule a sequence of tasks that will run consecutively, one after the other, once the previous task has been completed. This enables you to create more advanced processes on your data, without manual intervention, such as running aggregations over other aggregations, and conditioning alerts to only run once an integration has been updated.

This feature is available to project admins.

Settings up a new job

  1. From the project menu, select Jobs.
  2. Click the + icon to open a new job page.
  3. Enter a name (can use any name).
  4. One by one, select the tasks you want to run in the job. You can select from:
    • Segments
    • Aggregation Tables
    • Integrations
    • Alerts
    • Publications
  5. Once you’ve selected the type of task, choose one from the list of existing tasks in your project.
  6. You can re-arrange the order of the tasks by dragging the task icon up or down.
  7. Next, set the job schedule. Note that by adding a task to a job, you reset the tasks schedule to Manual. This means the task will only run on the job’s schedule. You can change the individual task’s schedule later from the task page.
  8. Lastly, if you’d like to be notified by email on any failed runs of this job, enter the emails to include in the list below. You can always check for run status in the job’s history, from the jobs list.
  9. Click Save to save the job, or Save and run now to also execute it. Once saved, click Run now to manually execute the job.

 

Managing saved jobs

The jobs list shows you all existing jobs.

From here you can:

  • Edit jobs: click any row to open and edit the job.
  • Delete, run now, and see the job history: from the row options (ellipses icon at the right end).

The job history shows all runs of the job. Click a row to see details of the tasks in the run. Successful runs will show all tasks it included. Failed runs stop at the first failed tasks, and will therefore show all tasks up to the one that failed.

 

Print Friendly, PDF & Email

Users

Managing user permissions

You can manage user permission to your project from the Project – Users page.

Add a user to the project

  1. Click the + (Add) button.
  2. Enter the user’s email and permission level. Note that “Viewer” permission can only be given to users who do not have any other type of permission yet, and vice versa.
  3. Click Save.
  4. New users will receive an email invitation to complete sign up – will show “Pending invitation” until done. You can resend the invitation by editing the user.
  5. Existing users’ permission will be updated on their next visit (or after page reload).

Modify a user permission level

  1. Click the user in the users list.
  2. Choose the new permission level. Note that “Viewer” permission can only be given to users who do not have any other type of permission yet, and vice versa.
  3. Click Update. The permission will be update on the next visit of the user (or after page reload).

See the list of dashboards accessible by a user

  • Locate the user in the users list.
  • Click the options icon on the right side of the row (ellipses) and choose “Dashboards”.

Remove a user from the project

  1. Locate the user in the users list.
  2. Click the options icon on the right side of the row (ellipses) and choose “Delete”.
  3. Confirm the deletion. The user will lose permission immediately. Note that you cannot delete the actual user, but only remove him from the project. For more information contact your CSM.

 

Project permission levels

Users can have one of the following levels of permission:

  • Admin:
    • Can make any changes to the Project, including changing Permissions.
    • Can see, create, duplicate, edit and delete all the Reports in the project.
    • Can create new Dashboards.
    • Can only access Dashboards if shared with him.
    • Can only edit Dashboards if given Edit permission on the dashboard.
  • User:
    • Can see all the Reports in the project.
    • Can create/duplicate Reports.
    • Can only edit/delete his own Reports.
    • Can see the project Event Sources, Live Events (for implementation purposes).
    • Can see the project Schema.
    • Can see, create and duplicate Segments.
    • Can only edit/delete his own Segments.
    • Can’t see/edit any other project page/setting.
  • Viewer: 
    • Only has access to specific Dashboards that have been shared with him.
    • Can change dashboard Date Range.
    • Can see dashboard Filter Report and select values if the filter is set to allow it.
    • Can’t see dashboard Global filters in the top bar or in the reports preview – this allows you to filter the dashboard by any sensitive information, hidden from viewers.
    • Can’t see the project Schema
    • Can’t see/create Reports not in dashboard.
    • Can’t edit any Reports.
    • Can’t access any Project page (menu is hidden).

Note on viewer permission:

A user can only have viewer permission if he does not have any other permission level on any project, and vice versa. To give viewer permission to a user that has already signed up, either use a different email or remove his permission from all other project first. Contact your CSM for additional assistance.

 

Full permissions list

FeatureActionAdminUserViewer
ReportsAdd newYesYesNo
ReportsDuplicateYesYesNo
ReportsViewYesYesNo
ReportsEditYesOnly ownNo
ReportsDeleteYesOnly ownNo
DashboardsAdd newYesYesNo
DashboardsDuplicateOnly own/sharedOnly own/sharedNo
DashboardsViewOnly own/sharedOnly own/sharedOnly shared
DashboardsEditOnly own/shared as "Edit"Only own/shared as "Edit"No
DashboardsDeleteOnly own/shared as "Edit"Only own/shared as "Edit"No
Dashboards permissionsShare/edit/delete permissionsOnly own/shared as "Edit"Only own/shared as "Edit"No
Dashboards date rangeViewYes, saved only for the user/when first sharedYes, saved only for the user/when first sharedNo
Dashboards date rangeEditYes, saved only for the user/when first sharedYes, saved only for the user/when first sharedNo
Dashboards filtersViewYes, saved only for the user/when first sharedYes, saved only for the user/when first sharedNo
Dashboards filtersEditYes, saved only for the user/when first sharedYes, saved only for the user/when first sharedNo
Dashboards filter reportViewYesYesYes
Dashboards filter reportEdit valuesOnly own/shared as "Edit" dashboards, or if filter is set to allow itOnly own/shared as "Edit" dashboards, or if filter is set to allow itNo
Dashboards filter reportEdit conditionsOnly own/shared as "Edit" dashboardsOnly own/shared as "Edit" dashboardsNo
Active projectViewYesYesNo
Active projectEditYesYesNo
Active projectAdd newYesYesNo
SchemaView YesYesNo
Event SourcesView YesYesNo
IntegrationsAdd newYesNoNo
IntegrationsViewYesNoNo
IntegrationsEditYesNoNo
IntegrationsRunYesNoNo
IntegrationsDeleteYesNoNo
Linked Data SourcesAdd newYesNoNo
Linked Data SourcesViewYesNoNo
Linked Data SourcesEditYesNoNo
Linked Data SourcesRunYesNoNo
Linked Data SourcesDeleteYesNoNo
SegmentsAdd newYesYesNo
SegmentsViewYesOnly ownNo
SegmentsEditYesYesNo
SegmentsRunYesOnly ownNo
SegmentsDeleteYesOnly ownNo
Aggregation TablesAdd newYesNoNo
Aggregation TablesViewYesNoNo
Aggregation TablesEditYesNoNo
Aggregation TablesRunYesNoNo
Aggregation TablesDeleteYesNoNo
AlertsAdd newYesNoNo
AlertsViewYesNoNo
AlertsEditYesNoNo
AlertsRunYesNoNo
AlertsDeleteYesNoNo
PublicationsAdd newYesOnly "Send now" from reportsNo
PublicationsViewYesNoNo
PublicationsEditYesNoNo
PublicationsRunYesNoNo
PublicationsDeleteYesOnly unsubscribe himselfOnly unsubscribe himself
JobsAdd newYesNoNo
JobsViewYesNoNo
JobsEditYesNoNo
JobsRunYesNoNo
JobsDeleteYesNoNo
Live EventsAdd newYesNoNo
Live EventsViewYesNoNo
Live EventsEditYesNoNo
Live EventsRunYesNoNo
Live EventsDeleteYesNoNo
EventsAdd newYesNoNo
EventsViewYesNoNo
EventsEditYesNoNo
EventsRunYesNoNo
EventsDisableYesNoNo
PropertiesAdd newYesNoNo
PropertiesViewYesNoNo
PropertiesEditYesNoNo
PropertiesRunYesNoNo
PropertiesDeleteYesNoNo
UsersAdd newYesNoNo
UsersViewYesNoNo
UsersEditYesNoNo
UsersRunYesNoNo
UsersDeleteYesNoNo
SettingsViewYesNoNo
SettingsEditYesNoNo
Print Friendly, PDF & Email

Events

What is an Event?

CoolaData collects user behavior data in the form of events.
An event is an action performed on your site/app.
Each event must be associated with the user who performed it, the time it happened, and the name of the event, and can include additional properties.

For each event, you can manage the following meta data:

  • Event name: The event name as sent to CoolaData and used in queries. Can use any characters, including spaces and capital letters. Event names are not case sensitive, so ‘Login’ and ‘login’ will be treated as the same event.
  • Category: Logical association. Can be used in queries to query multiple events simultaneously. Can be modified at any time and does not affect data storage.
  • Include in path: The path is used for CoolaData’s behavioral functions. Events not in path will not be included in these functions. Disable this only for events you want to exclude from behavioral analysis.

Please note the maximum amount of events per project is 1500 events.

Tracking events

CoolaData enables you to track your users’ behavior by sending events (comprised of properties).

The following is an example of an event sent to CoolaData as a JSON:

{
    "event_name": "deposit_server",
    "transaction_id": "63219",
    "session_os": "PC",
    "depositamount": "100",
    "pearlsamount": "0",
    "blackpearlsamount": "0"
}

The only mandatory field that you must enter in an event is event_name, (which in this example has the value “deposit_server”). An event name describes the event that occurred. You can specify any event name, for example: item_open, search_performed, game_level_3_achieved. All other properties are optional – add as many as you want, according to your needs.

In addition to the properties that you include, each CoolaData Tracker (SDK) automatically enriches each event with various properties. The following shows the same event (as shown above) as it arrives at CoolaData, after a CoolaData tracker automatically added (enriched) the user ID and the event’s timestamp:

{
    "user_id": "123456",
    "event_time_ts": "1467763537524",
    "event_name": "deposit_server",
    "transaction_id": "63219",
    "session_os": "PC",
    "depositamount": "100",
    "pearlsamount": "0",
    "blackpearlsamount": "0"
}

Best practice:

  1. Create a spreadsheet listing the events to be sent to CoolaData, including the event name and a description of each event.
  2. For each event, describe the properties to be included in that event, including its property name and a description. Remember – any property can be sent with any event.
    For example, we recommend creating a single button_clicked event that has a button_name property, rather than creating a separate event for each button clicked, such as: button_ A_clicked, button_ B_clicked, button_ C_clicked,

Mapping modes

CoolaData provides two modes for controlling the automatic generation of events and properties.

  • Self-learned – This is the default option. This means that all event names and all their properties are accepted by CoolaData (and are not defined as invalid). After each event and its properties are automatically defined by CoolaData, all subsequent events must arrive with properties that have the same data type. Otherwise, they are defined as invalid and not entered into CoolaData. We recommend that initially you start working with CoolaData using this option. Afterwards, you can start using the Lock option, described below.
  • Locked: This option enables you to control which event names and event properties are accepted by CoolaData. This means that only previously defined events and properties are considered valid, whether they were Self-learned or defined by you.
    • New Events Are Invalid – New events are not accepted and are stored in an Invalid Events table which you can review.
    • New Properties Are Not Accepted – New properties in an event are not accepted into CoolaData. The event itself (including all its other properties) is accepted into CoolaData. However, the new properties are not accepted – they are simply dropped.
    • Different Data Types Are Not Accepted – Properties with a different data type than originally defined are not accepted into CoolaData. All these events are listed in an Invalid Events table with a description that you can review.

To change the mapping mode:

  1. Go to Project – Settings
  2. Choose either Self-Learned or Lock mode
  3. Click Apply to save the change

Best Practice:

Either control all the events and properties or start by collecting them and filter them later:

  1. Full control: this option gives you complete control over which events and properties are valid in CoolaData, meaning all other events are not accepted by CoolaData and will appear in the Invalid Events list.
    1. Change your project mapping mode to Lock.
    2. Plan which events and properties will be considered valid.
    3. Define the valid events.
    4. Define the valid properties.
  2. Collect and filter: with this option CoolaData automatically generates events and properties from incoming events. You then get rid of the unwanted ones and lock down (prevent) the generation of new events and properties.
    1. By default, CoolaData operates in Self-learning mode.
    2. Send a few samples of each kind of event to CoolaData. CoolaData will automatically define these events and their properties as valid.
    3. Review all the automatically generated events and properties and then disable all unwanted events and delete all unwanted properties.
    4. Contact your CoolaData customer success representative to change to Lock mode.
    5. Notify all CoolaData integrators not to send the disabled events or the deleted properties because they will be sent to the Invalid Events list.

Defining an Event

When working in Lock mode, define your events using the events management interface.

  1. Open Project – Events.
  2. Click the Add button in the top right of the window to add a new event:
    4-3
  3. Fill in the window as follows and click the Save button.
    • Name (Mandatory): Describe the event that occurred in lowercase without spaces. For example, item_open, search_performed, game_level_3_achieved and so on.
    • Category: Select the value Other.
    • Active: Active is the default. If needed, deselect this option to specify that events with this Name (described above) are not loaded into CoolaData (Not Active). Non Active events are not listed in CoolaData’s Invalid Events list.
    • Include in Path: A path defines a specific sequence of events in the same user session. CoolaData provides various visualization options showing the path of events in a user session, such as a Funnel report or Cohort report. Only events for which this option is selected 4-4 are included in these visualizations.
      For example, an event that you may not want to include is a keep_alive event that verifies the devices’ connection.
      Regardless of the value selected in the Include in Path field (meaning even if it is off), the event still affects the session_duration (which is the duration of the session, in milliseconds) and is still counted in the session_event_counter (which is the number of events in the session).

Event Timestamps

CoolaData provides these options for specifying the timestamp of an event:

  • Device’s Timestamp (Default) – The CoolaData Tracker automatically adds the device’s timestamp.
    CoolaData automatically enriches each event with an event timestamp (named event_time_ts). This timestamp is determined by the device’s timeclock and indicates when the event happened, meaning when it was sent by your app using the CoolaData Tracker. CoolaData automatically stores and presents event timestamps in UTC time standard. The CoolaData Tracker also automatically adds a timezone_offset property to each event specifying the device’s time zone offset from UTC time, in hours. For example, -2.5.
  • Your Own Timestamp (Optional) – You can add your own timestamp to the event which overrides the device’s timestamp (described above).
    To specify your own timestamp for each event, simple add the event_time_ts property to the event’s JSON. Specify this timestamp in 13-digit format (milliseconds). CoolaData then uses this timestamp instead of your device’s (event_time_ts – described above).
    When using JavaScript Tracker or CoolaData REST API  in order to include your own timestamp, simple include the event_timestamp_epoch property (instead of event_time_ts – described above) in the event’s JSON. CoolaData then uses this timestamp which is UTC time standard.
    This option is only relevant when CoolaData is not set to use server timestamp,.
  • CoolaData Server Timestamp (Optional) – CoolaData assigns its own server’s timestamp to all events.
    In general the timeclocks of some devices’ may not be accurate. Upon request, CoolaData can configure your system so that CoolaData automatically assigns the timestamp of the CoolaData server to each event instead of the timestamp of the device that sent the event. To activate this feature, contact your CoolaData customer success representative.
    Benefit – The advantage of using this feature is to ensure that all the collected data is synchronized to the same clock, and that no events are rejected because of invalid dates.
    When this feature is activated, all timestamps that you send with the event are ignored and an accurate UTC timestamp is assigned instead. If you would like to use this feature and would also like to send your own timestamp, then add your own timestamp to each event in another property. For example, you could name it client_timestamp.

CoolaData automatically performs a sanity check of all event timestamps. By default, an event is considered Invalid when its timestamp is more than 12 hours into the future or 30 days into the past. For example, future timestamps may occur when an end user’s clock is set incorrectly. These types of events are not stored in CoolaData and appear in the Invalid List with the following reason – Future event sent or Past event sent.

Print Friendly, PDF & Email