Fivetran offers an automated data movement platform that simplifies the process of data integration by automating the extraction, loading, and transformation (ELT) of data across various cloud data platforms. The typical implementation process for Fivetran software is designed to be straightforward and efficient, catering to both small startups and large Fortune 500 companies.
Implementation Process
1. Initial Setup: The process begins with a simple setup that can be completed in as little as five minutes. This involves creating an account on the Fivetran platform and setting up the initial data connections. Fivetran provides over 500 no-code connectors that facilitate easy integration with various data sources.
2. Configuration: After the initial setup, users configure their data pipelines according to their specific business needs. This includes selecting the data sources and destinations, configuring data transformation rules, and setting up data replication schedules. Fivetran’s platform supports both log-based and log-free change data capture (CDC), which allows for real-time data movement with minimal impact on the source systems.
3. Automation and Scaling: Fivetran automates the most time-consuming parts of the ELT process, enabling data engineers to focus on more strategic tasks. The platform also offers features for workflow automation and programmatic scaling via the Fivetran API, which helps in tracking data movement and managing data flows efficiently.
4. Security and Compliance: Ensuring data security and compliance is a critical part of the implementation process. Fivetran provides robust security features, including advanced role-based access control, automated user provisioning, and metadata sharing with data catalog integrations. These features help protect data in-flight from source to destination and support data democratization and self-service analytics.
5. Optimization and Maintenance: Once the Fivetran platform is fully implemented, ongoing optimization and maintenance are carried out to ensure efficient operation. This includes monitoring the data pipelines, adjusting configurations as needed, and updating the connectors to handle new data sources or changes in the data infrastructure.
Implementation Duration
The duration of the implementation process can vary depending on the complexity of the data environment and the specific requirements of the business. However, due to the automated nature of the platform and the availability of no-code connectors, the initial setup can be completed very quickly, often within minutes. The subsequent phases of configuration, scaling, and optimization are also streamlined thanks to the platform’s automation features, which significantly reduce the time and effort required compared to traditional data integration methods.
Fivetran
By Fivetran
1. Custom Connector Framework: Fivetran provides a framework for building custom connectors using serverless functions. This allows businesses to create their own data pipelines within a broader ETL (Extract, Transform, Load) framework. The custom connectors are managed by clients, not Fivetran, which means businesses have the flexibility to tailor the connectors according to their specific data integration needs.
2. Development Skills Required: To develop a custom connector, developers need to be proficient in handling API documentation, managing HTTP codes, and writing code to build the integration. Technologies such as AWS Lambda, Azure Functions, and Google Cloud Functions are used to manage the infrastructure, reducing the complexity of building data infrastructure.
3. Security and Data Handling: When building custom connectors, developers must ensure the security of the data by managing authentication and data transfer securely over HTTPS. Fivetran supports this by calling the custom function via an HTTPS request, ensuring continuous protection of the data.
4. Integration with Cloud Platforms: Custom connectors need to be set up on one of the cloud platforms that Fivetran integrates with, such as Google Cloud Functions, Microsoft Azure Functions, or Amazon AWS Lambda. This integration allows the custom functions to be called by Fivetran to sync data to the destination.
5. Error Handling and Monitoring: Developers need to write custom logic to understand and address errors programmatically. Fivetran offers the ability to expose errors into the Fivetran dashboard if this logic is added to the function, helping in monitoring and troubleshooting.
6. Cost Considerations: Building and running custom connectors involve costs related to Fivetran services and cloud charges associated with running the serverless functions. It is important to consider the volume of data being processed and the efficiency of the code, as these factors directly impact cloud compute costs.
7. Support and Maintenance: While Fivetran does not maintain custom connectors for clients, companies like Portable offer services to build and maintain these connectors, providing an alternative for businesses that require custom integrations but lack the internal resources to develop them.
Fivetran's pricing model is primarily based on usage, specifically, the number of Monthly Active Rows (MAR) processed, which means the costs are largely variable depending on the volume of data integrated. However, there are several additional costs and considerations that businesses should be aware of when using Fivetran:
1. Setup Fees: Fivetran does not typically charge explicit setup fees for standard configurations. The platform is designed to facilitate easy setup without additional costs. The initial setup and configuration are included in the usage-based pricing model.
2. Maintenance Costs: Since Fivetran is a fully managed service, the maintenance costs are generally included in the subscription fee. Fivetran automates the data integration process, reducing the need for ongoing maintenance efforts from the user's side. This automation includes updates and maintenance of connectors, which Fivetran manages.
3. Support Charges: Fivetran offers different levels of support based on the subscription plan chosen. For example, the Enterprise plan includes priority support, while lower-tier plans might offer standard support. If a business requires more extensive support, such as dedicated personnel or faster response times, this could incur additional costs. The Business Critical and Enterprise plans, which offer the highest levels of support, are tailored for organizations that need robust data protection and compliance, and these plans might come with higher costs reflecting the enhanced support services.
4. Customization and Professional Services: If a business requires custom solutions or integration services beyond the standard offerings, Fivetran provides Professional Services at an additional cost. This might include custom connector development or specialized data integration strategies. These services are typically negotiated on a case-by-case basis and would add to the overall cost.
5. Data Transfer and Storage Costs: While not directly billed by Fivetran, transferring data from sources to destinations can incur costs from cloud service providers, such as AWS, Google Cloud, or Azure. These costs are based on the volume of data transferred and the specific pricing models of the cloud providers.
6. Compliance and Security Features: Plans that include advanced security and compliance features, such as the Business Critical plan, may cost more. These plans provide enhanced capabilities like private networking options and customer-managed keys for encryption, which are essential for businesses with stringent data security and compliance requirements.
7. Overage Charges: If the data usage exceeds the allocated Monthly Active Rows (MAR) under a specific plan, Fivetran may charge for the additional data processed. It's important for businesses to monitor their data usage closely to manage costs effectively.
8. Discount Opportunities: While not a direct cost, it's worth noting that Fivetran offers promotional discounts, such as those for Y Combinator startups or through negotiated enterprise agreements, which can significantly reduce overall expenses.
Fivetran offers a range of training and support options to new users to ensure they can effectively use the platform and maximize its capabilities.
Training Options
1. Online Training and Certification: Fivetran provides online training that includes basic and advanced courses. Users can progress from basic training to sales or technical certification. This training is available online, allowing users to learn at their own pace.
2. Customizable Templates and Guides: For new users, Fivetran offers customizable templates and guides to help them get started. These resources are designed to simplify the onboarding process and are particularly useful during the initial setup phase.
3. Udemy Course: There is also a course available on Udemy titled "Fivetran ETL from Scratch," which covers the basics to advanced aspects of using Fivetran. This course is suitable for various roles including ETL architects, data integration developers, and data engineers. It provides a comprehensive learning path from data integration to data migration using Fivetran.
Support Options
1. Community Support: Fivetran has a community platform where users can interact with other Fivetran users, ask questions, and share insights. This community is a valuable resource for new users to learn from experienced peers and find solutions to common challenges.
2. Technical Support: Fivetran offers technical support through its support team. Users can submit support tickets for more specific or technical issues they encounter. This support is crucial for troubleshooting and resolving issues that users cannot handle on their own.
3. Documentation and Resources: Fivetran provides extensive documentation, including setup guides, API documentation, and troubleshooting tips. This documentation is designed to help users understand how to use the platform and resolve issues independently.
4. Partner Program: For businesses that require more structured support, Fivetran's partner program includes training and certification for partners, which can be a resource for Fivetran users as well. Partners are trained and certified to assist Fivetran users, providing an additional layer of support.
5. 30-Day Success Guide: For organizations just starting with Fivetran, there is a 30-day success guide available on the Fivetran blog. This guide provides a structured approach to onboarding and maximizing the use of Fivetran within the first month.
These training and support options are designed to accommodate a variety of learning preferences and support needs, ensuring that new users can effectively integrate Fivetran into their data management workflows and leverage its full potential.
Fivetran employs a comprehensive set of security measures to protect data, ensuring that it remains secure both in transit and at rest.
Data Encryption
Key Management
Ephemeral Keys and HSM-Backed Keys: Fivetran uses a combination of ephemeral keys and hardware security module (HSM)-backed keys to enhance the security of the data transiting through its systems.
Compliance and Certifications
Compliance with Major Standards: Fivetran is compliant with major security and privacy standards, including SOC 1 Type 2, SOC 2 Type 2, ISO 27001, PCI DSS Level 1, and HIPAA. These certifications demonstrate Fivetran's commitment to maintaining high security standards.
Secure Data Handling and Operations
Data Handling Policies: Fivetran has strict policies for handling data securely, including the use of secure software development lifecycle practices and rigorous access controls.
Advanced Security Features
Anomaly Detection and Monitoring: Fivetran employs log-based monitoring and anomaly detection systems to identify and respond to threats in real-time, enhancing the ability to protect against potential security breaches.
Access Controls
Role-Based Access Control (RBAC): Fivetran implements RBAC to manage who has access to what data within an organization, ensuring that users only have access to the data necessary for their roles.
Multi-Factor Authentication (MFA): Access to critical systems is protected by MFA, adding an additional layer of security to prevent unauthorized access.
Security Protocols and Incident Response
These security measures collectively ensure that Fivetran provides a secure data integration environment, capable of meeting the needs of organizations with high security and compliance requirements.
Fivetran frequently updates its connectors and platform to enhance functionality, improve performance, and maintain compatibility with data sources. The management and deployment of these updates are structured to ensure minimal disruption to users while keeping the data integration processes up to date.
Frequency of Updates
1. Regular Updates: Fivetran releases updates on a regular basis, which includes both major and minor changes. These updates can be enhancements to existing connectors, introduction of new features, or routine maintenance updates.
2. Connector-Specific Updates: Updates for specific connectors are released as needed. These updates may address issues such as API changes from the data source, improvements in data handling, or security patches. For example, the NetSuite SuiteAnalytics connector and the Salesforce connector have seen multiple updates over the years to enhance functionality and performance.
3. Monthly Product Updates: Fivetran also sends out a monthly customer newsletter that highlights product and connector releases, providing users with the latest information on what has been updated or introduced.
Management of Updates
1. Automated Rollouts: Updates are typically rolled out automatically to all users. This ensures that all customers benefit from the latest improvements and security patches without having to manually update their systems.
2. Gradual Deployment: For significant updates, Fivetran often uses a gradual rollout process. This approach helps in monitoring the impact of the update and ensuring compatibility across all customer environments. For instance, major changes to the Salesforce connector were rolled out gradually over a period of three months.
3. User Notifications and Documentation: Fivetran keeps its users informed about updates through release notes, blogs, and direct communications. Detailed documentation is provided for each update, explaining the changes made and any actions users may need to take, such as updating their configurations or checking compatibility.
4. Version Management: For its HVR (High Volume Replicate) product, Fivetran supports each major release for a specific period, providing regular and extended support phases. This structured support lifecycle helps users plan for upgrades and ensures they are not caught off-guard by end-of-support dates.
5. Feedback and Support: Fivetran encourages feedback on its updates and provides support through its customer service channels. This feedback loop helps Fivetran address any issues quickly and adjust future updates to better meet user needs.
Fivetran's policy on data ownership and portability emphasizes that customers retain ownership and control over their data throughout the duration of their subscription to Fivetran services.
Data Ownership
- Customer Ownership and Control: Fivetran clearly states that the customer retains ownership of and control over the Customer Data in their account. This principle is fundamental to Fivetran's data handling policies, ensuring that customers maintain authority over their data.
Data Portability
- Data Portability Rights: Fivetran supports data portability, which is a critical aspect of data management under privacy regulations like the GDPR. Data portability allows customers to retrieve their data in a format that can be transferred to another system, facilitating ease of movement and use of data across different platforms or services.
- Compliance with GDPR: Fivetran's adherence to GDPR includes supporting the right to data portability. This compliance ensures that Fivetran facilitates the transfer of data at the request of the customer, aligning with the GDPR's requirements for data controllers and processors to provide data in a commonly used and machine-readable format.
These policies are part of Fivetran's broader commitment to data protection and privacy, ensuring that customers can manage their data effectively and in compliance with relevant regulations.
Fivetran offers flexible scaling options to accommodate changes in organizational needs, allowing businesses to adjust their data integration strategies as their requirements evolve.
Scaling Up
1. Usage-Based Pricing: Fivetran's pricing model is primarily based on the volume of data processed, measured in Monthly Active Rows (MAR). This model allows businesses to scale up their usage as needed without changing their subscription plan. As data volume increases, the cost per row decreases, making it cost-effective to scale up.
2. Adding Connectors and Destinations: Businesses can add more connectors or destinations to their Fivetran configuration without needing to change their subscription plan. This flexibility supports scaling up operations as organizational data needs grow.
3. Enterprise and Business Critical Plans: For organizations needing to scale up significantly, Fivetran offers Enterprise and Business Critical plans, which include advanced features like real-time data syncing, enhanced security options, and priority support. These plans are designed to meet the needs of large-scale operations.
Scaling Down
1. No Long-Term Commitments for Monthly Plans: Fivetran's monthly subscription plans do not require long-term commitments, allowing businesses to scale down by changing their plan or reducing their data usage without significant penalties. This is particularly useful for organizations that experience seasonal fluctuations or temporary decreases in data integration needs.
2. Downgrading Plans: Businesses can downgrade their subscription plans if they find that a lower-tier plan suffices their needs. This process can be managed through the Fivetran dashboard, where users can view and change their current plan.
3. Managing MAR Consumption: Organizations can also scale down by optimizing their data usage. This involves managing the number of Monthly Active Rows by removing unnecessary data syncs or optimizing data queries to reduce the volume of data processed.
Contractual Terms
- Automatic Renewals: Except for monthly plans, Fivetran subscriptions typically renew automatically for successive one-year periods unless a cancellation is requested. This clause is important for organizations to consider when planning long-term data strategy and budgeting.
- Cancellation Policies: Fivetran allows for plan cancellations, which can be processed through the user dashboard. This feature supports businesses in scaling down by discontinuing services they no longer need. However, it's important to note that cancellations can take up to 10 minutes to be effective.
Fivetran's terms and conditions for contract renewal and cancellation are designed to offer flexibility while also ensuring that customers are aware of their commitments.
Contract Renewal
1. Automatic Renewal: Except for monthly plans, Fivetran subscriptions automatically renew for successive one-year periods based on the terms outlined in the customer's current Order Form. This automatic renewal ensures uninterrupted service but requires customers to be mindful of their renewal dates if they wish to make changes.
2. Renewal Notification: While specific details on renewal notifications are not explicitly mentioned, it's common practice for SaaS companies to notify customers in advance of an upcoming renewal. Customers are encouraged to stay in communication with their Fivetran account managers for specific renewal terms and notifications.
Cancellation
1. Monthly Plans: Customers on monthly pay-as-you-go (PAYG) plans have the flexibility to cancel their plans at any time. This offers a high degree of flexibility for businesses with variable data integration needs or those wishing to trial Fivetran services without long-term commitments.
2. Cancellation Process: To cancel a plan, customers can navigate to the "Manage plan" section within their Fivetran dashboard and select the "Cancel Plan" option. Note that cancellations can take up to 10 minutes to be effective and reflect on the customer's dashboard.
3. Effect of Cancellation: Upon cancellation, it's implied that the service will cease, and any data processed through Fivetran will no longer be updated. However, specifics regarding data retention or deletion post-cancellation are not detailed and may require direct communication with Fivetran support.
4. Annual Contracts: Customers with annual contracts may face different conditions for cancellation, typically involving early termination fees or the fulfillment of the contract term before cancellation can be effective. These terms are likely detailed in individual Order Forms and may vary by customer.
5. No Refunds for Early Cancellation: Fivetran's policy does not explicitly mention refunds for early cancellation of annual contracts. It's common in SaaS agreements that customers remain responsible for the full contract value even if they choose to cancel early.
6. Contract Modifications: Customers looking to modify their contracts, either by scaling up or down, can do so by contacting Fivetran support or their account manager. Modifications may involve transitioning to a different plan or adjusting the volume of Monthly Active Rows (MAR) based on usage needs.
7. Grace Period for Overages: For customers exceeding their usage limits under certain plans, Fivetran provides a one-time grace period to allow for plan adjustments or usage optimization. This policy helps prevent service interruptions due to unexpected overages.
Fivetran software meets a comprehensive range of compliance standards, ensuring that it adheres to rigorous security and privacy requirements across various industries and regions.
1. SOC 1 Type 2 and SOC 2 Type 2: Fivetran undergoes annual independent audits for SOC 1 Type 2 and SOC 2 Type 2, which assess the effectiveness of the company's controls related to security, availability, and confidentiality.
2. ISO/IEC 27001:2013: Fivetran has achieved ISO 27001 certification, which is a globally recognized standard that specifies requirements for an information security management system (ISMS). This certification demonstrates Fivetran's commitment to maintaining high levels of information security.
3. PCI DSS Level 1: Fivetran has been validated for PCI DSS Level 1, the highest level of certification under the Payment Card Industry Data Security Standard. This is particularly important for customers who handle cardholder data.
4. HIPAA: Fivetran complies with the Health Insurance Portability and Accountability Act (HIPAA) requirements for Protected Health Information (PHI) and is willing to sign a Business Associate Agreement (BAA) with customers who are subject to HIPAA mandates.
5. GDPR: Fivetran is compliant with the General Data Protection Regulation (GDPR), ensuring that it meets EU data protection requirements. This includes mechanisms to respond to data subject requests and to report data breaches in compliance with GDPR timelines.
6. HITRUST: Fivetran has attained HITRUST Implemented, 1-year (i1) Certification, which provides assurance of the company's information security controls, particularly in managing data protection and mitigating cybersecurity threats.
7. EU-U.S. Privacy Shield and Swiss-U.S. Privacy Shield: Although Fivetran ceased relying on these frameworks as a legal basis for international data transfers following the Schrems II decision, it continues to adhere to the principles of these frameworks for all personal information transferred to the U.S. prior to July 16, 2020.
These certifications and compliance measures ensure that Fivetran adheres to strict standards for data security and privacy, making it a reliable choice for organizations concerned with regulatory compliance across various sectors.
