[{"data":1,"prerenderedAt":33},["ShallowReactive",2],{"post-data-fireworks-notion-ai-incident-rotate-secrets":3},{"post":4,"relatedPosts":17},{"id":5,"title":6,"content":7,"hashtags":8,"coverImage":13,"createdAt":14,"updatedAt":14,"seoTitle":15,"seoDescription":16},"KNeYAWi0hGQ8OEYpHzJR","Fireworks AI and Notion AI Incident: Which Secrets to Rotate If You Pasted Them Into AI Chats or Eval Jobs","\u003Cdiv class=\"container\">\n\u003Cp>If you stored secrets in Fireworks evaluation jobs, or pasted passwords or API keys into Notion AI, treat those credentials as exposed and rotate them at the service that issued them. Fireworks AI says an attacker reached environment variables in evaluation jobs and later more customer data. Notion says its Insights data held at Fireworks included Notion AI chat transcripts, and it revoked Notion API keys it found in them on November 11, 2026. Neither company has said the keys were misused. Rotation is cheap and does not depend on that answer.\u003C\u002Fp>\n\u003Cp>This article is based on the vendors' own incident pages, opened on November 12, 2026, plus press coverage. We did not test any product and have no access to the affected data. The incident is still under investigation, so details may change.\u003C\u002Fp>\n\u003Ch2>What happened, in order\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Short answer:\u003C\u002Fstrong> a breach at an AI inference vendor first touched secrets in evaluation jobs, then a development environment, and later turned out to include a dataset from a customer, Notion.\u003C\u002Fp>\n\u003Cp>Facts below come from \u003Ca href=\"https:\u002F\u002Ffireworks.ai\u002Fsecurity\u002Fbulletins\u002Foctober-2026-security-incident\">Fireworks' bulletin\u003C\u002Fa>, its \u003Ca href=\"https:\u002F\u002Ffireworks.ai\u002Fsecurity\u002Fbulletins\u002Foctober-2026-security-incident-update\">November 9 update\u003C\u002Fa> and \u003Ca href=\"https:\u002F\u002Fwww.notion.com\u002Fen-us\u002Fsecurity\u002Fadvisories\u002Fsecurity-incident-involving-a-third-party-ai-vendor\">Notion's advisory\u003C\u002Fa>. Fireworks times are PDT.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>October 6, 9:00 am:\u003C\u002Fstrong> Fireworks identified the activity and began containment. An unauthorized third party had used internal Fireworks credentials and accessed \"environment variables used in evaluation jobs\" in a limited number of customer accounts. Fireworks believes API keys and tokens stored as secrets were captured in some of them.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>October 9, 10:00 pm:\u003C\u002Fstrong> Fireworks published its bulletin. Affected customers had been notified directly.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>October 23-24:\u003C\u002Fstrong> per the update, a third-party cloud provider that hosted Fireworks R&D machines had its own security incident. An unauthorized party used it to reach Fireworks development machines and ran an unauthorized workload, causing additional customer data impact.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>November 2:\u003C\u002Fstrong> Fireworks told Notion that malicious actors had accessed a small set of data. Notion first found only synthetic testing data.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>November 6-8:\u003C\u002Fstrong> Fireworks said customer data may have been accessed, then confirmed the final dataset. Notion found it included Notion AI chat transcripts used for its Insights feature.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>November 9:\u003C\u002Fstrong> Notion contacted impacted customers. Fireworks updated its bulletin; status: active monitoring.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>November 11:\u003C\u002Fstrong> Notion revoked exposed Notion API keys found in affected user messages.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2>What is confirmed and what is not\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Short answer:\u003C\u002Fstrong> the vendors confirm that secrets and chat transcripts were reached. They do not confirm misuse, counts, or complete scope.\u003C\u002Fp>\n\u003Cp>Confirmed by Fireworks: resource metadata, billing details (addresses, invoice history, balance), and, for a limited set of accounts, secrets, datasets and incomplete model weights. Payment card numbers were not stored and not affected. Fireworks says Inference, the Training SDK and Nexus, which are zero data retention by default, showed no impact. It describes the findings as ongoing, saying the investigation \"may reveal additional information\".\u003C\u002Fp>\n\u003Cp>Confirmed by Notion: fewer than 1% of workspaces are affected. For most of them the transcripts did not identify the author or workspace and contained no tool call results. Notion says no other features were affected.\u003C\u002Fp>\n\u003Cp>Not stated in the pages we read:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>how many keys or workspaces are involved;\u003C\u002Fli>\n\u003Cli>whether any exposed key was used by the attacker;\u003C\u002Fli>\n\u003Cli>which third-party cloud provider was breached;\u003C\u002Fli>\n\u003Cli>whether the scope will grow.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Fireworks links the activity to a financially motivated group that sells unauthorized LLM access, which the update calls \"LLMJacking\". That is Fireworks' attribution, and we have not verified it independently.\u003C\u002Fp>\n\u003Cp>One caution on dates: some press coverage dates the first bulletin October 10. The bulletin itself shows publication on the evening of October 9 Pacific time, which is already October 10 in UTC.\u003C\u002Fp>\n\u003Ch2>Do you need to act?\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Short answer:\u003C\u002Fstrong> act if you received a notice, or if you ever put a credential where these vendors could hold it. Both vendors say that if you have not heard from them, they have not identified your account or workspace as affected.\u003C\u002Fp>\n\u003Cp>Notion says that if you have not heard from it, it has not identified your workspace as affected and no action is needed right now. Fireworks says the same about its affected-customer notice.\u003C\u002Fp>\n\u003Cp>Even so, a short self-check costs little:\u003C\u002Fp>\n\u003Col>\n\u003Cli>Did you ever store API keys or tokens as secrets in Fireworks evaluation jobs or other Fireworks resources?\u003C\u002Fli>\n\u003Cli>Did anyone on the team paste a password, key or connection string into Notion AI during the incident window?\u003C\u002Fli>\n\u003Cli>Did you get an email from either company?\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>If any answer is yes or unsure, continue.\u003C\u002Fp>\n\u003Ch2>Rotate in this order\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Short answer:\u003C\u002Fstrong> replace revoked keys, then rotate every third-party credential that sat in the exposed place, then review activity. Removing a stored secret does not invalidate it.\u003C\u002Fp>\n\u003Ch3>1. Fireworks API keys\u003C\u002Fh3>\n\u003Cp>Fireworks revoked identified keys, including ones the attacker created. Its guidance: create new keys if still needed and update applications, which fail authentication until updated. Check for unrecognized API keys, users, evaluators, training jobs and datasets, and report unfamiliar names or IDs to Fireworks.\u003C\u002Fp>\n\u003Ch3>2. Third-party credentials that lived in Fireworks secrets\u003C\u002Fh3>\n\u003Cp>Fireworks tells affected customers to rotate the credentials named in their notice at the issuing provider, then update applications and Fireworks secrets. As its bulletin puts it, removing a stored secret does not invalidate the credential itself. Fireworks says it can share secret names and credential fingerprints through a secure channel, which helps you know which ones to rotate.\u003C\u002Fp>\n\u003Cp>Rotate even if provider logs look clean. Fireworks' guidance is to rotate them even if you do not find suspicious activity.\u003C\u002Fp>\n\u003Ch3>3. Anything pasted into Notion AI\u003C\u002Fh3>\n\u003Cp>Notion advises: if you pasted passwords, API keys or other credentials into Notion AI during the incident timeframe, rotate them. Because transcripts may be unattributable, you cannot count on the vendor to tell you whose secret it was. Notion's own API keys that appeared in affected messages were revoked on November 11, but Notion's page does not say how many, so replace any Notion token that was ever pasted into a chat.\u003C\u002Fp>\n\u003Cp>For a Notion internal connection token, Notion's \u003Ca href=\"https:\u002F\u002Fdevelopers.notion.com\u002Fguides\u002Fauthorization\u002Fcredential-security\">credential guidance\u003C\u002Fa> documents this path: Settings, Connections, Develop or manage connections, find the connection, select Refresh. Then update every application using the old token, test, and remove the old value from config files and documentation. The same page advises checking recent changes to pages and databases and looking for connections you do not recognize.\u003C\u002Fp>\n\u003Ch3>4. Review activity at each provider\u003C\u002Fh3>\n\u003Cp>For every rotated credential, look at the issuing provider's audit log for unfamiliar reads, downloads, new keys or new users between early October and the rotation. Do this after you have rotated, not before. Waiting for evidence can keep a working key in the attacker's hands.\u003C\u002Fp>\n\u003Ch2>Why keys end up in places like this (And How to Fix It)\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Short answer:\u003C\u002Fstrong> convenience. An evaluation job needs an API key, and an assistant works better when you paste the failing config. Each of those places is now a vendor's storage, and you inherit that vendor's incident.\u003C\u002Fp>\n\u003Cp>Two habits turn this incident from a notice into an emergency:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>Credentials as chat input.\u003C\u002Fstrong> A key pasted into an AI assistant is stored as a transcript. You cannot rotate a transcript, only the key.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Long-lived keys with wide scope.\u003C\u002Fstrong> A key that works from anywhere and never expires is worth stealing. A key limited to one job and a short lifetime loses most of its value when it leaks.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Practical changes that do not depend on any one vendor:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Give AI tools the shape of a config, not the values. Replace secrets with placeholders such as \u003Ccode>API_KEY=<redacted>\u003C\u002Fcode> before pasting into Notion AI or ChatGPT.\u003C\u002Fli>\n\u003Cli>Keep an inventory: for each secret, where it is used and how to rotate it. Without it, a vendor notice that names secrets is hard to act on.\u003C\u002Fli>\n\u003Cli>Turn on secret scanning in repositories and CI, as Notion's guidance also recommends.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cdiv class=\"highlight-box\">\n\u003Ch3>Action: Use Ennote for Zero-Persistence Evaluation Jobs\u003C\u002Fh3>\n\u003Cp>Instead of manually storing credentials in Fireworks evaluation jobs or saving them to local \u003Ccode>.env\u003C\u002Fcode> files, you should inject secrets dynamically at runtime.\u003C\u002Fp>\n\u003Cp>By using the \u003Cstrong>Ennote CLI\u003C\u002Fstrong>, you can stream API keys directly into a job's volatile memory. For example: \u003Cbr>\u003Ccode>ennote secret get fireworks-eval-keys -- <your-eval-script>\u003C\u002Fcode>\u003C\u002Fp>\n\u003Cp>When the job finishes, the credential evaporates entirely. It never rests on disk, and it is never permanently saved into the third-party vendor's configuration storage, entirely bypassing the vulnerability exploited in this incident.\u003C\u002Fp>\n\u003C\u002Fdiv>\n\u003Ch2>For teams\u003C\u002Fh2>\n\u003Cp>A short plan you can run today:\u003C\u002Fp>\n\u003Col>\n\u003Cli>Ask the team, without blame, who pasted credentials into Notion AI or stored secrets in Fireworks. People remember if the question is safe.\u003C\u002Fli>\n\u003Cli>Collect the names in the Fireworks notice, if you got one, and the list from step 1.\u003C\u002Fli>\n\u003Cli>Rotate in the order above, one service at a time, and verify each deployment before revoking the old key.\u003C\u002Fli>\n\u003Cli>Record what you rotated and when. If an auditor or customer asks, you will have an answer.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Migrate away from persistent configs.\u003C\u002Fstrong> Adopt a Zero-Persistence architecture like Ennote so that your team's infrastructure secrets stream directly to local processes or Kubernetes deployments without ever ending up saved in an AI vendor's system.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Ch2>What to watch next\u003C\u002Fh2>\n\u003Cp>Both pages say the investigation is ongoing. Things that would change the advice: a named third-party cloud provider, evidence of misuse of the exposed keys, or a wider dataset. Re-read the \u003Ca href=\"https:\u002F\u002Ffireworks.ai\u002Fsecurity\u002Fbulletins\u002Foctober-2026-security-incident-update\">Fireworks bulletin\u003C\u002Fa> and the \u003Ca href=\"https:\u002F\u002Fwww.notion.com\u002Fen-us\u002Fsecurity\u002Fadvisories\u002Fsecurity-incident-involving-a-third-party-ai-vendor\">Notion advisory\u003C\u002Fa> before closing your own incident.\u003C\u002Fp>\n\u003Ch2>Sources\u003C\u002Fh2>\n\u003Cul>\n\u003Cli>Fireworks AI: \u003Ca href=\"https:\u002F\u002Ffireworks.ai\u002Fsecurity\u002Fbulletins\u002Foctober-2026-security-incident\">October 2026 security incident\u003C\u002Fa> (last updated October 10) and \u003Ca href=\"https:\u002F\u002Ffireworks.ai\u002Fsecurity\u002Fbulletins\u002Foctober-2026-security-incident-update\">update of November 9\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>Notion: \u003Ca href=\"https:\u002F\u002Fwww.notion.com\u002Fen-us\u002Fsecurity\u002Fadvisories\u002Fsecurity-incident-involving-a-third-party-ai-vendor\">Security incident involving a third-party AI vendor\u003C\u002Fa>, last updated November 10 with a November 11 entry\u003C\u002Fli>\n\u003Cli>Notion developers: \u003Ca href=\"https:\u002F\u002Fdevelopers.notion.com\u002Fguides\u002Fauthorization\u002Fcredential-security\">Credential security\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>Press: \u003Ca href=\"https:\u002F\u002Fthehackernews.com\u002F2026\u002F11\u002Ffireworks-ai-breach-widens-to-expose.html\">The Hacker News\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fcryptoslate.com\u002Fnotion-users-put-access-keys-in-ai-chats-a-supplier-breach-exposed-them\u002F\">CryptoSlate\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>Not done: we did not test any rotation step, did not see the customer notices, and did not independently verify Fireworks' attribution. All pages were opened on November 12, 2026 through summarizing fetches, so exact wording should be checked against the pages before quoting.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Chr>\n\u003Cp>\u003Cstrong>Keep work secrets out of chats and job configs.\u003C\u002Fstrong> Ennote is a secret manager for passwords, API keys and environment variables, with role-based access, an audit log and Kubernetes sync. \u003Ca href=\"https:\u002F\u002Fapp.ennote.io\u002F?utm_source=blog&utm_medium=article&utm_campaign=fireworks-notion-ai-incident-rotate-secrets&utm_content=footer\">Start for free\u003C\u002Fa> on the Developer plan.\u003C\u002Fp>\n\u003C\u002Fdiv>",[9,10,11,12],"FireworksAI","NotionAI","CyberSecurity","ZeroPersistence","https:\u002F\u002Ffirebasestorage.googleapis.com\u002Fv0\u002Fb\u002Fblog-01-c712e.firebasestorage.app\u002Fo\u002Fblog-covers%2F1794508698548_Gemini_Generated_Image_ddks6nddks6nddks.jpeg?alt=media&token=d87147d8-d914-4ff0-b2ea-349545dc6f4c",1794508700056,"Fireworks AI & Notion Incident: What to Rotate","Fireworks AI's incident exposed secrets in eval jobs and Notion AI chats. Learn what is confirmed, and why Ennote's zero-persistence model prevents this.",[18,23,28],{"title":19,"slug":20,"coverImage":21,"createdAt":22},"The Evolution of Kubernetes Secret Delivery (And Why Polling is Dead)","the-evolution-of-kubernetes-secret-delivery-and-why-polling-is-dead","https:\u002F\u002Ffirebasestorage.googleapis.com\u002Fv0\u002Fb\u002Fblog-01-c712e.firebasestorage.app\u002Fo\u002Fblog-covers%2F1777523487463_diagramm%20(2).jpg?alt=media&token=5b01f974-cfe5-4890-b68f-5e178b422c6e",1777523490608,{"title":24,"slug":25,"coverImage":26,"createdAt":27},"mSecure Password Manager Incident: What Is Confirmed and What to Reset First","msecure-breach-what-to-do","https:\u002F\u002Ffirebasestorage.googleapis.com\u002Fv0\u002Fb\u002Fblog-01-c712e.firebasestorage.app\u002Fo\u002Fblog-covers%2F1793932710332_Gemini_Generated_Image_r7yeg4r7yeg4r7ye.jpeg?alt=media&token=f42c9c4d-8c71-4615-9a80-97f67b94bdea",1793903987316,{"title":29,"slug":30,"coverImage":31,"createdAt":32},"Platform Breach: How to Rotate All Your Environment Variables","rotate-environment-variables-after-platform-breach","https:\u002F\u002Ffirebasestorage.googleapis.com\u002Fv0\u002Fb\u002Fblog-01-c712e.firebasestorage.app\u002Fo\u002Fblog-covers%2F1793652683825_Gemini_Generated_Image_oj1iemoj1iemoj1i.jpeg?alt=media&token=41e54608-2ae0-40d9-acd8-192d54c0f1c9",1793652685659,1794508774523]