PRIVATE AI 101

Public AI isn’t for personal or proprietary data

PUBLIC AI

Lost data sovereignty

Lost confidentiality, privacy, IP

Black-box, opaque decisions

Generic interface, one LLM

High cost, variable billing

Overweight models over-process

ZAI NODE'S PRIVATE AI

Completely private, sovereign

Doesn’t train, no sharing

Fully transparent, auditable

Custom AI infra & apps

Lower cost, more certainty

Fit for purpose models & infra

Private vs public AI

  • Private AI is an approach to AI designed with data protection at its core. Instead of sending sensitive data to public clouds and open LLMs for training and processing, private AI keeps information within your infrastructure — whether that’s on-prem, in a VPC, or inside secure containers.

    Private AI = Local AI

    Public AI = Open-sourced AI

  • Your data (inputs, prompts, attachments, uploads, outputs, answers, documents) doesn’t leave the machine. Our solution works without being connected to the internet or cloud.

  • There are thousands of software providers who claim to keep data private, but what they really mean is that they de-identify personal information (PII) before the data goes online, to the LLM provider or to the cloud. These techniques may solve PII but they don’t protect other private, third party or confidential info or IP.

    1. Data ownership: private AI ensures that your sensitive corporate and personal data, and IP remain within your control. This reduces the risks associated with sharing data with external entities.

    2. Regulatory compliance: private AI solutions comply with data protection laws such as GDPR, HIPAA, GLBA, the Privacy Act. Adhering to these regulations is vital to avoid legal consequences.

    3. Tailored solutions: private AI allows for customised AI models tailored to your organisation's requirements. Unlike public AI, which may offer generic solutions, private AI enables you to develop models specifically optimised for your business needs.

    4. Cost effectiveness: private AI offers more predictable costs, especially at scale. By investing in your own AI infrastructure and data management, you can avoid the unpredictable expenses associated with public AI services.

    5. Strategic independence: with private AI, you maintain strategic independence over your AI initiatives. By controlling your data and AI models, you can make decisions that align with your organisation's long-term goals and objectives.

    6. Audit trail: ensure that your AI remains fully auditable and accountable.

    1. All data intake and preprocessing stays in your environment.

    2. Model is trained locally using centralised or federated learning.

    3. Privacy first inherent via encryption and anonymisation.

    4. Deployed in controlled, offline infra according to your policies.

    5. Audit and accountability — full visibility of model behaviour.

    1. Data ownership: full control of data.

    2. Built-in compliance and regulatory compliance.

    3. Faster, smarter AI: local responsiveness with localised response.

    4. Lower risk: no data leaks or shadow AI.

    5. Scalable architecture: adaptable for hybrid/multi-cloud.

    1. More complicated IT integration.

    2. Higher skilled technology team requirements.

    3. Governance policies required.

    4. Infrastructure investment.

  • For each client, Zai Node creates a web application where, after users submit their data (queries, documents etc.), the data remains accessible exclusively within the client's infrastructure, visible only to them and the users they authorise.

    • Cost predictability is a key concern as enterprises move workloads from public to private cloud environments.

    • Also a desire for greater control and security is leading enterprises to devote more resources to private cloud infrastructure. See this research from Broadcom for more info.

    • It is theoretically possible for a cloud provider to view all data that passes through their systems.

    • The more important concern with cloud providers is:

      • Cost, particularly long-term cost stability.

      • Availability of additional GPU compute when scaling needs hit.

      • Being vendor locked and then above two points are triggered.

    • See also CIO.com and Tech Radar.

  • Public Private

    Data control Limited Full

    Privacy risk High Low

    Customization Low High

    Deployment Web Local

    Compliance Lacking Built-in

    Cost Variable Fixed

Testing vendor claims of data security & privacy

  • In the fine print, most vendors say they “may not use the data for training”. This is highly unlikely because learning from the data is the primary method to improve LLMs.

    Most software vendors don’t own the LLM so ultimately the LLM provider controls what data they use for training.

  • Any data sent anywhere these days is encrypted, so this is a moot point from vendors.

    Whilst it’s common for data to be encrypted before being sent, the data is decrypted at the final destination (the public LLM owner and/or application vendor). Vendors cannot claim that encrypted data is never decrypted and never visible to any system. If so, the system would not function at all.

  • What they are really saying is, 'We actually don't run any of our AI ourselves, and most likely don’t really understand AI, so we use this [big tech] LLM and cloud company. We also have no idea how they do data protection, so go look it over at their policies and leave us alone.’

    Then see recent AWS system fail.

  • The AI app/software vendor must see the client data to operate on the data in some way. If they can’t see client data, then they are simply passing it onto another provider to operate on it in some way. It is extremely unlikely that a vendors’ platform is not storing client data. It would be required to be stored for it to be surfaced in the UI of the app or otherwise mutated in their platform.

    If an AI app uses a public LLM, the LLM provider must be able to see the data to produce answers for the data or operate on it in some way. Most companies using external LLM providers dance around this, but, ultimately, they are passing their clients’ data onto third parties for processing.

  • This may be correct, however it widely known that public LLM providers share submitted data with third party processors, partners and LE/regulators. They also use the data for training new models. See here.

  • Some vendors simply run public LLMs on the cloud, and connect them up to a basic chat app. This is more “private” than using the public LLM directly, but still the client data is going to the cloud and being processed by someone else — often the cloud owned by the same big tech company who operates the LLM. Thus, it's not private AI and it's definitely not sovereign AI.

    • Claim 1: “Your data and materials never leave Australia.”

    • Claim 2: “We don’t use foreign cloud storage. There’s no offshore access. Geographic restrictions prevent offshore access.”

    • Claim 3: “We will ensure you comply with Australian Data Privacy laws.”

    • Claim 4: “If someone breaches our system, your data is encrypted and unreadable.”

    • Claim 5: “We exclusively use [Big Tech] Cloud & AI in Australia, and we rely on their infrastructure.”

    • Issue 1: The US CLOUD Act says that US data companies must provide stored data for a customer or subscriber on any server they own and operate anywhere in the World, when requested.

    • Issue 2: Once data is in another organisation’s infra, especially when it is encrypted, the vendor has no idea where it goes.

    • Issue 3: Data is not protected. There is no current way to provide LLMs with encrypted data.

    • Also check: where is the vendor’s backend database? A simple IP geo search may show an offshore DB.

    • Issue 4: with [Big Tech] storage and LLM, and offshore DB, there’s no way client data doesn’t leave Australia.

Other questions? ask us