Secure Databricks Deployments for Regulated Industries

Banks and insurers have more data than ever. Yet turning it into value remains hard:

• Core systems, CRM, payments, claims and partner data sit scattered across dozens of disconnected applications.
• Data teams wait for data they can fully trust.
• Reporting, audit and data-residency requirements keep tightening.
• AI-generated fraud and cyberattacks evolve faster than siloed monitoring can follow.

And all of this has to happen under constant cost pressure, with scarce data and AI talent.

The result: underused data, AI pilots that never reach production, and a false choice between compliance gaps and aging legacy systems.

A modern data platform can change that. But in a regulated environment, the question is never only what the platform can do. It is how it is deployed, secured and operated.

Databricks is one of the leading data and AI platforms. It covers the full chain, from data processing and analytics to machine learning and agentic AI (AI systems that can plan and carry out tasks on their own). The platform is evolving quickly, with recent additions such as a managed Postgres database for both analytical and transactional workloads, and an AI Gateway to control and monitor access to AI models.

For regulated organizations, three aspects matter most:

• Enterprise-grade security features that allow secure integration at several layers: network, identity and authorization, and application security.
• An open architecture based on open formats and technologies, which reduces vendor and strategic lock-in.
• One platform for data and AI, meaning one place to integrate data and break silos, and one place to govern access instead of many.

ELCA has deployed Databricks for several financial institutions. For example, a Swiss cantonal bank began its cloud journey in 2023, focusing initially on data, and built a secure data analytics platform on Databricks. This platform has undergone several security audits. We also support a major insurer in Lausanne in modernizing its data integration and AI capabilities, as well as several private banks and asset managers in their data and AI platform initiatives.

We specialize in complex, highly secured deployments, where every data access and every data share must be precisely defined, validated and documented. For organizations that prefer to focus on their data products rather than on operations, our Data Platform as a Service (DPaaS) offers a Databricks platform operated by Everyware (ELCA group), hosted in Switzerland.

Technology is only part of the answer. A successful deployment starts by understanding the available skills, the requirements and the context. It ends with a clear target state: the operating model, the training and upskilling of the teams, and a hypercare period (a phase of reinforced support after go-live) with defined duration and goals.

Compliance frames the whole journey. Upstream, a risk analysis defines the expected security measures, in line with regulatory requirements and data protection laws such as the Swiss FADP (nLPD) and the GDPR. Downstream, audits and penetration tests verify that the measures actually in place match what was promised.

Once the vision is set, a high-level definition of the platform is required. It must answer a few structuring questions:

• What is the platform governance? Who, in business and IT, does what during the project and in operations.
• Where must data reside? Where the data is stored and processed depends on regulatory and internal requirements. This topic is even more complex for AI platforms that rely on hosted LLM.
• How many environments? Typically development, quality assurance and production, each with its own level of protection.
• How many independent workspaces? Separate workspaces can isolate highly confidential data or give business units their own space.
• How does it fit into the existing landscape? The platform must integrate with the network layout and security (firewalls, proxies, data leakage prevention), with the software delivery process, and with application security standards.
• What is the transition plan? How long will the build take? Is it replacing an existing platform? Are there large data volumes to ingest or migrate?

These early decisions shape everything that follows. Changing them later is costly.

Databricks moves fast. More and more capabilities are delivered as serverless or managed services, where the provider runs the infrastructure, for example AI models for inference. This brings agility, but it also extends the architecture and the security perimeter beyond the organization’s own network. Each new service must be assessed, configured and monitored before it is opened to users. Security is therefore not a one-time project, but a continuous practice.