Only this pageAll pages
Powered by GitBook
1 of 79

Unistream Docs

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Cloud Deployment

Platform Overview

Getting Started

Home

System Requirements

Deployment Options

Visual Modeling

Core Components

Data Integration

Quick Start Guide

Security & Governance

On-Premise Deployment

Data Writeback

Report Generation

Data Mart

Integrated Functionality

Data Warehouse

Object Generation Methods

Creation Process

Automatic Optimizations

Schema Management

Object-Level Security

Unihub

Dashboard Creation

Performance Monitoring

BI System

Data Encryption

Data Compression

V25.10.10

New

  • Writeback page updated with Lake component UI

    The Writeback page now uses the Lake component’s UI, providing a more consistent and user-friendly experience across the platform.

Improved

  • "Migrated" checkbox now works correctly in the Lake component.

  • Refresh functionality fixed and now works in Audit Logs.

  • Added new button animations to enhance user interaction feedback.

Bug Fixes

  • Fixed ClickHouse integration bugs.

V25.12.17

New

  • Alert Notifications: Receive real-time alerts for important events and updates.

  • Enhanced Error Messages: New popup dialogs provide clearer and more actionable error information.

V25.10.10

We're excited to introduce Unihub - your centralized hub for managing your organization's Unistream platform usage, subscriptions, and administrative tasks.

Key Features

  • Usage Tracking: Monitor and analyze your organization's Unistream platform usage in real-time

Role Management

ETL Models

ETL Processes

Resources

Creation Process

Data Storage & Optimization

Release Notes

Use Cases

Unistream BI

Access Control

Security Considerations

Row-Level Security

AI Module

Implementation

Glossary

Community Support

Clickhouse DBMS

Mobile Access

Integrations

FAQ

Audit & Monitoring

Token Usage Cards: Detailed cards display comprehensive information about your token consumption.

  • Automatic Token Purchase: Enable automatic purchase of additional premium tokens when your balance runs low.

     Token Cost Breakdown: View detailed information about token costs for each action and  operation.

  • Insurance & Retention Information: New informational sections help you understand insurance and data retention policies.

  • Bug Fixes

    • Fixed pagination issues across pages.

    • Resolved diagram popup display problems.

    • Fixed filter functionality on the Logs page.

    • Corrected UI bugs in the API keys management section.

    Invoice Management: Store, view, and download invoices for your Unistream subscription

  • API Key Management: Securely generate, view, and manage API keys for Unistream integrations

  • Subscription Management: View and manage your organization's subscription plan, billing details, and service levels

  • What You Can Do

    • View detailed usage metrics and analytics for your Unistream instance

    • Access and download invoices

    • Update subscription preferences and billing information

    • Monitor usage against your subscription limits

    DataGrip

    This guide provides step-by-step instructions for connecting DataGrip to Unistream.

    Connection Steps

    1. Click on Copy connection string in Unistream's Integration hub

    2. In DataGrip's Database Explorer (shortcut: Alt+1 on Windows/Linux or ⌘+1 on macOS), click on the + sign

    3. Select Import from Clipboard

    4. Provide your password in the popup window

    5. Click on Download drivers if prompted

    You are now connected to Unistream's database.

    Default Settings

    • Socket timeout: 5 Minutes

    • Connection mode: Read Only

    • SQL Quotation: Double quotes (")

    Video Tutorial

    For a visual walkthrough, watch our tutorial video:

    DbVisualizer

    This guide provides step-by-step instructions for connecting DbVisualizer to Unistream.

    Connection Steps

    1. Download the unistream.jar file from Unistream's Integration Hub

    2. Open DbVisualizer

    3. Click on File → Import Settings

    4. Locate and select the downloaded unistream.jar file

    5. A popup window will appear showing the Unistream connection in the list

      • Click OK

    6. DbVisualizer will restart automatically

    7. After restart, in the Databases panel, you will see the Unistream connection

      • Double-click on the connection

    8. In the connection dialog:

      • Your username (email) will be automatically detected

      • Enter your Unistream password

    9. Click Connect

    10. Accept driver downloads if prompted

    You are now connected to Unistream's database through DbVisualizer.

    Video Tutorial

    For a visual walkthrough, watch our tutorial video:

    V25.10.20

    New

    • Added number formatting options in transformation operations.

    Improved

    • Enhanced column mapping validation to skip removed fields, reducing unnecessary validation errors.

    • Improved UI/UX on the Minds page for better user experience and navigation.

    Bug Fixes

    • Fixed diagram rendering and display issues.

    • Resolved visual bugs affecting subflow components.

    • Fixed filter functionality in mind sources.

    • Corrected various UI inconsistencies across the platform.

    QStudio

    This guide provides step-by-step instructions for connecting TimeStored's Qstudio to Unistream.

    Connection Steps

    1. In Qstudio, click on Server in the top navigation panel

    DBeaver

    This guide provides step-by-step instructions for connecting DBeaver to Unistream.

    Connection Steps

    1. Download the JSON configuration file from Unistream Integration Hub

    2. Navigate to your DBeaver workspace folder using one of these methods:

    Release Notes

    API Reference

    test

    test2

    Performance Tuning

    Monitoring

    Select Add Server
  • In the connection dialog:

    • For Server Type, select Clickhouse.com

    • For Connect By, select URL

    • Paste the URL copied from Unistream's Integration Hub

    • Provide a Connection Name (can be any name you prefer)

    • Enter your Username (your Unistream email)

    • Enter your Password

  • Click Test Connection

  • Download drivers if prompted

  • Click Save

  • You are now connected to Unistream's database through Qstudio.

    Video Tutorial

    For a visual walkthrough, watch our tutorial video:

    • Run this command in Command Prompt:

      {% code overflow="wrap" fullWidth="true" %}

      {% endcode %}

    • Or paste this path directly in File Explorer:

      {% code overflow="wrap" %}

      {% endcode %}

  • Paste the downloaded JSON file in the opened folder

  • If DBeaver is already open:

    • Right-click on an empty space in the Database Navigator

    • Click "Refresh"

    If DBeaver was closed, simply open it

  • You will see the Unistream connection in your Database Navigator

    • Right-click on it

    • Select "Edit Connection"

  • In the connection settings window:

    • Provide your username (email) and password in the Authentication section (Database Native)

    • Click "Download drivers" if prompted

    • Click "Test Connection" to verify

    • Click "OK" to save

  • You are now connected to Unistream's database.

    Video Tutorial

    For a visual walkthrough, watch our tutorial video:\

    Jenkins Setup and Pipeline Configuration

    Requirements

    You will need a server running Ubuntu 24.04 or later with minimum specifications:

    • 4 CPU cores (16 CPU cores recommended)

    • 16GB RAM (64 GB RAM recommended)

    • 128GB storage

    • 30 Mbps internet speed

    • User with passwordless sudo access (explained in detail below)

    Note: The application is resource-intensive. More CPU cores (16+ cores) and especially more RAM (64GB+) are strongly recommended for optimal performance.

    Setup Passwordless Sudo Access

    Jenkins requires passwordless sudo access to execute deployment and system-level commands automatically, as it runs under a non-interactive service account.

    1. Edit the sudoers file safely:

    2. Add this line (replace "username" with your deployment user):

    3. Save and exit

    Jenkins Installation

    1. Update Packages

    2. Install Java

    3. Add Jenkins Repository Key

    4. Add Jenkins Repository

    5. Update Packages with Jenkins Repository

    6. Install Jenkins

    7. Start Jenkins Service

    Unistream Configuration

    1. Clone Repository

    Use "Unistream" as the username and the PAT token provided via email as the password

    2. Navigate to Directory

    3. Run Configuration Script

    The script will ask for VM host, sudo user, and password. This information is used for SSH connection. For VM host, specify the IP address of the same machine in the case of single VM deployment.

    4. Get Admin Password

    Jenkins Configuration

    1. Open {Jenkins host}:8080 in your browser

    2. Enter the admin password and complete the registration

    3. Go to Plugins → Available plugins

    4. Find and install the "Publish Over SSH" plugin

    Help

    For questions, please contact the datamind team.

    V25.10.1

    New

    • More intuitive icons for Excel file download buttons in Warehouse

      We’ve updated the icons for Excel file downloads to be more intuitive and visually aligned with the overall UI design, improving user clarity and consistency.

    • Unified table UI for WH and Lake modules

      We’ve implemented WH’s improved table design in both WH and Lake modules. This provides a consistent experience and better usability across both sections.

    Improved

    • Added warning messages on certain views to guide user actions and prevent common mistakes.

    • Enhanced table layout responsiveness.

    Bug Fixes

    • Fixed validation issues for dimension tables.

    • Resolved various UI misalignments and restored missing icons and buttons.

    • Fixed the minimap display in the Lineage section.

    • Fixed an issue where role icons were not displaying correctly in User Management.

    Google Analytics Service Account

    This guide provides step-by-step instructions for connecting Google Analytics 4 to Unistream.

    Prerequisites

    You will need:

    • A Google account with access to Google Cloud Console

    Power BI

    This guide provides step-by-step instructions for connecting Microsoft Power BI to Unistream.

    Prerequisites

    You will need:

    • Microsoft Power BI Desktop installed on your computer (recommended to install via Microsoft Store)

    @explorer "C:\Users\%USERNAME%\AppData\Roaming\DBeaverData\workspace6\General\.dbeaver"
    C:\Users\%USERNAME%\AppData\Roaming\DBeaverData\workspace6\General\.dbeaver

    Resolved minor column width inconsistencies in the Lake module.

    Process Creation

    Troubleshooting

    Column-Level Security

    Configuration Interface

    Single Sign-On

    Data Lake

    Scaling Strategies

    Agree to restart after installation
  • After restart, go to Manage Jenkins → Credentials

  • Update the "Unistream/******* (Azure DevOps PAT for Jenkins)" credential with the PAT provided via email

  • Find and run the "full-unistream-deployment" pipeline on the main page

  • sudo visudo
    username  ALL=(ALL:ALL) NOPASSWD: ALL
    sudo apt update
    sudo apt install openjdk-17-jre -y
    curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | \
    sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
    echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
    https://pkg.jenkins.io/debian-stable binary/ | \
    sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
    sudo apt update
    sudo apt install jenkins -y
    sudo systemctl enable jenkins 
    sudo systemctl start jenkins
    git clone -b main --single-branch --depth 1 \
    https://dev.azure.com/datamindge/Unistream/_git/prod
    cd prod
    sudo bash setup-jenkins-config.sh
    sudo cat /var/lib/jenkins/secrets/initialAdminPassword

    Admin or Editor access to your GA4 property

  • Unistream platform credentials

  • Setup Steps

    1. Create a Service Account

    1. Open Google Cloud Console: https://console.cloud.google.com/

    2. If you don't have a project, create one:

      • Click the project selector in the top left corner

      • Select "New Project"

      • Enter a project name and click "Create"

    3. Navigate to IAM & Admin → Service Accounts

    4. Click + CREATE SERVICE ACCOUNT

    5. Fill in the details:

      • Service account name: ga4-unistream

      • Service account ID: auto-generated

    6. Click DONE

    2. Generate JSON Key

    1. In the Service Accounts list, find your newly created account

    2. Click the three dots (⋮) → Manage keys

    3. Click ADD KEY → Create new key

    4. Select JSON and click CREATE

    5. The key file will download automatically — store it securely

    3. Enable APIs

    1. In Google Cloud Console, go to APIs & Services → Library

    2. Search for and enable the following APIs:

      • Google Analytics Data API

      • Google Analytics Admin API

    3. Click ENABLE for each API

    4. Grant Access in GA4

    1. Open Google Analytics: https://analytics.google.com/

    2. Go to Admin (gear icon in the bottom left)

    3. Select your Property

    4. Navigate to Property Access Management

    5. Click + → Add users

    6. Enter your Service Account email:

      • Find this in your downloaded JSON file under client_email

      • Format: [email protected]

    7. Select Role: Viewer

    8. Click Add

    5. Find Your Property ID

    1. In Google Analytics, go to Admin

    2. Select your Property

    3. Go to Property Settings

    4. Copy the Property ID (a numeric value, e.g., 123456789)

    ClickHouse ODBC driver

  • Unistream platform credentials (email and password)

  • Installation Steps

    1. Install ClickHouse ODBC Driver

    1. Download the ClickHouse ODBC driver from the official GitHub repository: https://github.com/ClickHouse/clickhouse-odbc/releases

    2. If Windows is preventing you from running the .msi file:

      • Unzip the zip file

      • Right-click on the .msi file → Properties

      • At the bottom of the Properties window, check the "Unblock" checkbox

      • Click "Apply" then "OK"

      • Now you can run the installer

    3. Install the driver by following the installation wizard instructions

    2. Download Connection Files from Unistream

    In Unistream's Integration Hub, you will find two download options:

    1. Direct Query & Import Objects (tables, views)

      • Uses native ClickHouse connection (still requires ODBC driver)

    2. Import Mode Custom SQL

      • Uses ODBC connection

      • Allows you to provide custom SQL queries

    3. Connect to Unistream

    1. Unzip the downloaded folder for your preferred connection method

    2. Open the .pbip file with Power BI Desktop

    3. When prompted for authentication (if not prompted, click "Refresh" in Power BI Desktop):

      • Enter your username (email)

      • Enter your password

    4. You are now connected to Unistream

    Additional Features

    For instructions on how to:

    • Toggle between connection modes

    • Import multiple tables

    • Change custom SQL queries

    Please refer to our video tutorial below.

    Video Tutorial

    For a visual walkthrough, watch our tutorial video:

    Excel

    This guide walks you through setting up the integration between Unistream and Microsoft Excel.

    Prerequisites

    You will need:

    • Microsoft Excel installed on your computer

    • Clickhouse ODBC driver

    • Unistream platform credentials (email and password)

    Installation Steps

    1. Install Clickhouse ODBC Driver

    1. Download the Clickhouse ODBC driver from the official GitHub repository:

    2. If Windows is preventing you from running the .msi file:

      • Unzip the zip file

      • Right-click on the .msi file → Properties

    2. Connect Excel to Unistream

    1. Open Microsoft Excel

    2. Navigate to the Data tab in the ribbon

    3. Click on Existing Connections

    4. Select Browse for More

    Working with Unistream Data in Excel

    Loading a Complete Table or View

    1. In Excel, click on Queries & Connections

    2. In the right panel, right-click on the connection named "The best Data Platform"

    3. Select Edit

    4. This will open the Power Query Editor

    Running Custom SQL Queries

    1. Follow steps 1-3 above to open the Power Query Editor

    2. Remove all steps except the Source step

    3. Click the settings icon (gear) on the Source step

    4. Replace the default query with your custom SQL

    Using Pivot Tables with Unistream Data

    You can create powerful data visualizations by using Pivot Tables with your Unistream connection:

    1. Select your data range

    2. Navigate to the Insert tab

    3. Click Pivot Table

    4. Choose your data source and placement

    Video Tutorial

    For a complete walkthrough of these features, watch our tutorial video

    Troubleshooting

    If you encounter connection issues:

    • Verify your Unistream credentials

    • Check that the Clickhouse ODBC driver is properly installed

    • Ensure your network allows connections to the Unistream platform

    • Contact your system administrator for assistance with firewall settings

    For additional support, please contact the Unistream support team.

    Connectors

    Orchestration

    Data Minds Overview

    At the bottom of the Properties window, you will see a checkbox labeled "Unblock"

  • Check this box and click "Apply" then "OK"

  • Now you can run the installer

  • Install the driver by following the installation wizard instructions

  • Locate and choose the Unistream_con.odc file you downloaded from the Unistream platform

  • Click OK

  • Enter your Unistream credentials:

    • Username: Your email address used for Unistream

    • Password: Your Unistream password

  • Click OK to establish the connection

  • In the Applied Steps panel on the right, click the settings icon (gear) on the navigation step

  • Select any other table from the dropdown menu

  • Click OK to load the selected table

  • Click OK to execute your custom query

    Build your pivot table using the Unistream data fields

    https://github.com/ClickHouse/clickhouse-odbc/releases

    Dimension & Fact Objects

    Data Synchronization Patterns

    Network Requirements and Domain Whitelist

    This document outlines the network configuration requirements for Unistream on-premise deployment, including port access requirements and domain whitelists for both Jenkins and Unistream machines.

    Domain Whitelist Requirements - Quick Reference

    Jenkins Server Domains

    Unistream Machine Domains

    Port Access After Deployment

    Administrator Access (Full Access)

    Administrators should have access to the following ports:

    Standard User Access

    Standard users should have access to the following ports:

    Domain Whitelist Requirements (Detailed)

    Jenkins Server Whitelist

    Last updated: 05-08-2025

    The Jenkins server requires access to the following domains:

    Domain Pattern
    Description

    Unistream Machine Whitelist

    Last updated: 05-08-2025

    The Unistream machine requires access to the following domains:

    Operating System & Package Management

    Domain Pattern
    Description

    Database Systems

    Domain Pattern
    Description

    Development Tools & Languages

    Domain Pattern
    Description

    Version Control & Source Code

    Domain Pattern
    Description

    Container & Cloud Services

    Domain Pattern
    Description

    External Libraries & Assets

    Domain Pattern
    Description

    Unistream Services

    Domain Pattern
    Description

    Help

    For questions, please contact the datamind team.

    dev.azure.com
    *.dev.azure.com
    *.visualstudio.com
    *.azuredevops.com
    github.com
    *.github.com
    *.jenkins.io

    *.jenkins.io

    Jenkins official website, plugin repository, and update services

    *.yarnpkg.com

    Yarn JavaScript package manager registry and binaries

    *.nodesource.com

    Node.js distribution and repository hosting

    *.microsoft.com

    Microsoft services and updates (for .NET, VSCode, etc.)

    dev.azure.com

    Azure DevOps services for repository access and pipeline integration

    *.dev.azure.com

    Azure DevOps subdomains and regional endpoints

    *.visualstudio.com

    Visual Studio Team Services and legacy Azure DevOps endpoints

    *.azuredevops.com

    Modern Azure DevOps services and APIs

    github.com

    GitHub main site for repository access and API calls

    *.github.com

    GitHub subdomains including API endpoints and CDN services

    *.ubuntu.com

    Ubuntu package servers, updates, and metadata

    *.canonical.com

    Canonical services (Ubuntu's parent company)

    *.debian.org

    Debian project infrastructure, mirrors, and updates

    *.launchpad.net

    Ubuntu development and PPA (Personal Package Archives) hosting

    *.launchpadcontent.net

    Content delivery for Launchpad (e.g., PPA package files)

    apt.postgresql.org

    PostgreSQL APT package repository

    www.postgresql.org

    Official PostgreSQL website and documentation

    *.clickhouse.com

    ClickHouse packages, documentation, and updates

    *.oracle.com

    Oracle database connection client downloads

    pypi.org

    Python Package Index – base domain

    *.pypi.org

    Python Package Index – for installing Python packages

    *.pythonhosted.org

    Hosts documentation and package metadata for PyPI

    bootstrap.pypa.io

    Hosts get-pip.py and other Python bootstrapping tools

    *.python-poetry.org

    Poetry Python dependency manager site and downloads

    *.npmjs.org

    NPM JavaScript package registry

    githubusercontent.com

    GitHub raw content and release assets

    *.githubusercontent.com

    GitHub content delivery network for repositories and assets

    *.docker.com

    Docker official site and associated resources

    *.docker.io

    Docker Hub and Docker CDN endpoints

    storage.googleapis.com

    Google Cloud Storage – used by various projects to serve binaries/assets

    *.amazonaws.com

    AWS-hosted services and S3 buckets

    *.cloudflarestorage.com

    Cloudflare-hosted storage/CDN content

    rsms.me

    Font hosting (e.g., Inter font used by many applications)

    cdn.sheetjs.com

    SheetJS library CDN for spreadsheet processing functionality

    *.unistream.cloud

    Unistream AI service and documentation service endpoints

    *.ubuntu.com
    *.canonical.com
    *.debian.org
    *.launchpad.net
    *.launchpadcontent.net
    apt.postgresql.org
    www.postgresql.org
    *.clickhouse.com
    *.oracle.com
    pypi.org
    *.pypi.org
    *.pythonhosted.org
    bootstrap.pypa.io
    *.python-poetry.org
    *.npmjs.org
    *.yarnpkg.com
    *.nodesource.com
    *.microsoft.com
    githubusercontent.com
    *.githubusercontent.com
    *.docker.com
    *.docker.io
    storage.googleapis.com
    *.amazonaws.com
    *.cloudflarestorage.com
    rsms.me
    cdn.sheetjs.com
    *.unistream.cloud
    8123, 5433, 8080, 8081, 8082, 1745, 7070, 6901, 6902, 6903, 6908
    8081, 8082, 1745