The typical implementation process for Upsolver software, with steps in order and brief explanations:
Initial Planning and Requirements Gathering
Define your analytics and AI data goals, identify data sources, and determine target destinations (e.g., data warehouses, Iceberg lakes).
Platform Setup and Account Creation
Sign up for Upsolver and configure your environment on AWS or your preferred cloud infrastructure.
Source and Destination Connection
Use Upsolver’s built-in connectors to link your operational databases, message queues, and other data sources, as well as your analytics or ML targets.
Template Selection or Pipeline Design
Choose from Upsolver’s expert-made templates for common use cases or design custom data pipelines tailored to your specific requirements.
Data Ingestion and Pipeline Configuration
Configure data ingestion from sources to your data lake or warehouse, setting up transformation, deduplication, and schema evolution as needed.
Automatic Table Optimization
Enable Iceberg Table Optimizer for continuous monitoring and optimization of tables, ensuring peak query performance and cost efficiency.
Testing and Validation
Run initial data flows, validate data quality, check for synchronization, and ensure all transformations and optimizations are working as expected.
Upsolver is highly customizable and designed to meet specific business needs. Here's a concise overview of its customization capabilities:
SQL-Based Transformations: Use standard SQL with streaming-specific extensions for custom logic.
Dual Interface: Switch between visual UI and code editor for flexible pipeline design.
Calculated Fields & Filters: Create custom fields and filters using built-in functions or Python code.
Custom Python Functions: Extend logic using Python in calculated fields.
REST API: Programmatically manage pipelines and integrate with CI/CD workflows.
Schema Evolution: Automatically adapts to schema changes with visual data profiling.
Flexible Outputs: Write data to various destinations like Athena, Redshift, Snowflake, etc.
Data Observability: Monitor data flow, quality, and changes in real time.
ironSource used Upsolver to build a multi-purpose data lake, automate transformations, and reduce storage costs—highlighting its adaptability to real-world use cases.
Upsolver offers a suite of training and support resources to assist new users in effectively utilizing their data integration platform. Here's an overview of the available options:
14-Day Free Trial: New users can initiate a 14-day free trial, which begins upon running the first job. This trial provides full access to Upsolver's features, allowing users to explore the platform's capabilities without restrictions.
Hands-On Training and Technical Consultation: During the trial period, Upsolver's team offers extensive support, including hands-on training sessions and technical consultations. This assistance helps users define their use cases and implement solutions effectively, ensuring readiness for production-scale operations.
On-Demand Online Training Course: Upsolver provides an on-demand online training course designed to help users get started and assist them throughout their journey with the platform.
Comprehensive Documentation: The platform offers extensive documentation covering various topics, including setup guides, connector configurations, job creation, and data observability. These resources are available at docs.
Video Tutorials: A collection of video tutorials is available, demonstrating tasks such as creating Amazon Athena outputs, parsing dates, and handling hierarchical data. These tutorials cater to different learning preferences and provide visual guidance on using Upsolver's features.
In-App Chat and Slack: Users can access real-time assistance through in-app chat and Slack channels, facilitating prompt support and collaboration.
Video Calls: For more in-depth discussions or complex issues, Upsolver offers support via video calls, allowing for personalized guidance and troubleshooting.
24/7 Phone Support: Critical issues are addressed through 24/7 phone support, ensuring continuous availability and rapid response times based on agreed-upon metrics.
Upsolver implements a comprehensive set of security measures to protect customer data, ensuring compliance with industry standards and providing flexibility to meet various organizational requirements.
1. Private VPC Deployment:
Data Residency: All data processing occurs within the customer's AWS account, ensuring that data never leaves their environment.
Access Control: Customers have full control over access permissions, leveraging AWS security primitives to restrict access based on IP addresses, VPNs, and other methods.
Isolation: Upsolver's control plane operates separately, and Upsolver employees do not have access to customer data.
2. Fully Managed Deployment:
Quick Setup: Upsolver manages the infrastructure, processing data in its own AWS account.
Data Handling: While processing occurs outside the customer's account, data is read from and written back to the customer's S3 buckets.
Security Measures: No data is stored on disk outside the customer's AWS account, and access is governed by strict permissions.
Data Storage: Upsolver does not store customer data. All data is stored in the customer's Amazon S3 buckets, ensuring that Upsolver processes data in-memory without retaining it.
IAM Roles and Permissions: Upsolver utilizes AWS Identity and Access Management (IAM) roles to define permissions. Two roles are created:
Management Role: Allows Upsolver to manage servers but does not grant data access.
Data Role: Grants access to data in S3 and Kinesis, but is only assumable within the customer's AWS account.
Access Restrictions: Customers can define allowed and disallowed IP addresses, and choose to permit or deny Upsolver support team access for troubleshooting purposes.
SOC 2 Compliance: Upsolver has implemented processes and automations to meet SOC 2 requirements, enhancing security and compliance practices. This includes improved credential management and user access controls. Scytale
Upsolver maintains a consistent and structured update schedule to enhance its platform's functionality, performance, and user experience.
Monthly Feature Releases: Upsolver publishes detailed feature summaries approximately once a month. Recent releases include:
August 2024 Feature Summary
July 2024 Feature Summary
June 2024 Feature Summary
May 2024 Feature Summary
Weekly Software Updates: In addition to monthly summaries, Upsolver implements weekly software updates that encompass new features, enhancements, and bug fixes. For example, in July 2023, updates were released on July 4th, 13th, 19th, 27th, and 31st.
Cloud-Based Deployment: As a cloud-based SaaS platform, Upsolver updates are deployed periodically from the cloud.
Gradual Rollout: Updates are rolled out gradually to ensure service stability and minimize potential disruptions.
Non-Disruptive UI Updates: User interface updates are designed to be non-intrusive. Users may receive an "Upgrade" prompt, allowing them to switch to the new UI version immediately. If the prompt is dismissed, the updated UI will be applied upon the next session refresh or reopening.
Upsolver's policies on data ownership and portability prioritize customer control, security, and compliance. Here's an overview:
Customer Data Control: Customers retain full ownership of their data. Upsolver processes data in-memory and does not store customer data on its servers. All data resides in the customer's Amazon S3 buckets.
Private VPC Deployment: For enhanced security, Upsolver can be deployed within a customer's Virtual Private Cloud (VPC). In this setup, even Upsolver employees cannot access the data, ensuring compliance with regulations like HIPAA and GDPR.
Standardized Formats: Upsolver supports data export in widely-used formats such as Parquet, facilitating seamless integration with other platforms and tools.
Upsolver's policies on contract renewal and cancellation are detailed in their Terms of Service. Here's a comprehensive overview:
Defined Term: Each Order Form specifies a start and end date, establishing the contract's duration. The agreement remains effective as long as there is an active Order Form.
Renewal: The Terms of Service do not explicitly state automatic renewal provisions. Therefore, renewal terms are likely outlined in individual Order Forms. Customers should review their specific Order Forms to understand renewal conditions.
Material Breach: Either party can terminate the agreement if the other commits a material breach and fails to remedy it within 30 days of receiving written notice.
Insolvency: The agreement may be terminated if either party becomes insolvent, files for bankruptcy, or undergoes similar proceedings not dismissed within 60 days.
Post-Termination Obligations:
All accrued but unpaid fees become immediately due.
Licenses and usage rights granted to the customer terminate.
Certain provisions, such as confidentiality and indemnification, survive termination.
Non-Cancellable and Non-Refundable: Once executed, Order Forms are non-cancellable and non-refundable, except as explicitly provided in the agreement.
Usage Commitments: Unused Upsolver Units expire at the end of their term without refund or credit.
Upsolver adheres to several industry-standard compliance frameworks and implements robust security measures to ensure data protection and regulatory compliance.
SOC 2 Type II Certified: Upsolver has achieved SOC 2 Type II certification, which evaluates the effectiveness of its security controls over time, ensuring the protection of customer data.
GDPR and HIPAA Compliance: Upsolver is designed to support compliance with the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA). When deployed in a private Virtual Private Cloud (VPC), even Upsolver employees cannot access customer data, aligning with stringent data privacy requirements.
Data Protection Addendum (DPA): The company offers a DPA that outlines its obligations concerning data processing and security, ensuring adherence to data protection laws.
Private VPC Deployment: Upsolver can be deployed within a customer's AWS account, ensuring that data processing occurs entirely within the customer's environment. This setup provides enhanced security and control over data.
Data Storage: Upsolver does not store customer data on its servers. All processed data is stored in the customer's Amazon S3 buckets, maintaining customer ownership and control.
