Firestore Emulator v1.22.0 reports ambiguous evaluation errors for valid false Rules predicates #11283

Activity

1. seryxdev

   added

   type: bug

   on Oct 14, 2026
2. aalej

   added

   emulators: firestore

   on Oct 14, 2026
3. sharanbingi commented on Oct 17, 2026

   

   sharanbingi

   on Oct 17, 2026

   More actions

   Additional evidence for #11283: the same ambiguous denial diagnostic occurs on an update comparing an existing immutable string field, in both Firestore emulator 1.20.2 and 1.22.0. This extends the reported create/false-map-predicate examples; it does not establish a common internal cause.

   Environment: Windows, Java OpenJDK25.0.3 (build25.0.3+-15898627-b508.16), rules-unit-testing3.0.4, Firebase JS SDK10.14.1 / Firestore SDK4.7.3. Installed CLI15.30.1; JARs launched directly. Synthetic demo project only; no production comparison.

   Exact tested guarded rule (single match, no recursive overlap):

   rules_version = '2';
   service cloud.firestore {
     match /databases/{database}/documents {
       function allowedTypedUpdate(dealer) {
         let user = get(/databases/$(database)/documents/users/$(request.auth.uid)).data;
         return request.auth != null && user.get('active', false) == true && user.get('deleted', false) != true &&
           user.get('role', '') == 'SALES_USER' && user.get('dealershipId', '') == dealer &&
           resource != null && request.resource != null &&
           resource.data.dealershipId == dealer && request.resource.data.dealershipId == dealer &&
           user.get('storeId', '') == resource.data.storeId && user.get('storeId', '') == request.resource.data.storeId &&
           resource.data.keys().hasAll(['saleId']) && request.resource.data.keys().hasAll(['saleId']) &&
           resource.data.get('saleId', '') is string && request.resource.data.get('saleId', '') is string;
       }
       match /dealerships/{dealer}/sales/{sale} {
         allow update: if allowedTypedUpdate(dealer) && request.resource.data.saleId == resource.data.saleId;
       }
     }
   }
   

   Synthetic authenticated UID alpha_sales_a; trusted /users/alpha_sales_a:

   {
     "uid": "alpha_sales_a",
     "role": "SALES_USER",
     "dealershipId": "dealership_alpha",
     "storeId": "STORE_A",
     "active": true
   }

   Existing /dealerships/dealership_alpha/sales/sale_seed_001:

   {
     "saleId": "sale_seed_001",
     "dealershipId": "dealership_alpha",
     "storeId": "STORE_A",
     "customerId": "cust_001",
     "vehicleId": "veh_001",
     "loanId": "loan_seed_001",
     "paymentType": "CASH",
     "finalSaleAmountPaise": 8500000,
     "amountReceivedPaise": 8500000,
     "outstandingAmountPaise": 0,
     "paymentStatus": "COMPLETED",
     "notes": "Initial seed notes"
   }

   Each operation independently reseeds this fixture. Both old and proposed saleId are present strings. No transforms; authenticated SDK .update().

   Operation	1.20.2	1.22.0
   {notes:'metadata-positive',updatedAt:2}	allowed; exact proposed data	same
   {saleId:'sale_seed_001'}	allowed; unchanged document	same
   {saleId:'forged_sale_id'}	denied; unchanged document; diagnostic below	same
   Forged ID after removing only final equality	unsafe mutation allowed	same

   Both guarded SDK responses (code: permission-denied) and emulator stderr contain exactly:

   evaluation error at L15:24 for 'update' @ L15, false for 'update' @ L15
   

   The SDK prefixes it with 7 PERMISSION_DENIED:. No version-dependent wording difference or expression-limit diagnostic was observed. L15 identifies the allow entrypoint, not a proven internal failing expression. The equality-free control is intentionally unsafe and is not a fix. No missing-field control was added to this reproduction.

   Loaded-rules evidence on both versions: observed local :securityRules PUT, HTTP200, actual guarded payload SHA256 ED0306B445E8AFA8DD49348C2AC028AB7A6CACBD88605B0C0DD0935FB5C52B4A. The observer checks the uploaded bytes, demo project and selected local port. Rule payloads, coverage, SDK errors, mutation control and both positive-control records are attached. Coverage is cumulative per predicate; it does not prove the internal cause. Post-read deep equality verified unchanged rejected documents and exact successful updates; raw after maps were not separately serialized.

   Both guarded runs exit1 under the preserved strict diagnostic assertion, despite correct functional denial. Both unsafe-control runs also exit1; a granted mutation is never counted as security acceptance. Portable harness copies only parameterize the port and have not been rerun after packaging; exact executed predicates/fixtures and result evidence are retained.

   Could maintainers confirm whether this false immutable-field comparison is another expected instance of the diagnostic ambiguity described here, and which supported diagnostic field distinguishes it from a genuine evaluator exception? Internal cause remains unverified; no production parity or broader rule correctness is claimed.
4. seryxdev commented on Oct 17, 2026

   

   seryxdev

   on Oct 17, 2026

   Author

   More actions

   Thank you for the additional reproduction and detailed comparison between Firestore Emulator versions 1.20.2 and 1.22.0.

   This is particularly relevant to our testing, where we are observing similar ambiguous evaluation error diagnostics during security-rule denial tests.

   In our isolated Firebase Emulator environment, we recorded:

   • 27 generic evaluation errors
   • 0 expression-limit warnings
   • 26/26 expected authorization outcomes in the B1 owner Rules suite

   Our strict diagnostic checks remain blocked because a PERMISSION_DENIED response containing evaluation error cannot automatically be treated as a clean, intentional denial.

   The new immutable-field reproduction suggests that some of these diagnostics may occur even when the intended authorization behavior is preserved. However, we cannot assume the same root cause without further verification.

   Could the Firebase team clarify:

   1. Is this diagnostic behavior expected when a Rules predicate legitimately evaluates to false, or does it indicate an evaluator defect?
   2. Is there a supported way to distinguish an intentional Rules denial from a genuine evaluation exception?
   3. Are there plans to improve the diagnostic reporting in a future Firestore Emulator release?
   4. Is there a recommended testing approach for security-sensitive applications while this issue remains unresolved?

   We are deliberately keeping our strict security assertions enabled and are not treating HTTP 403 responses alone as sufficient evidence of Rules correctness.

   Thank you for investigating this issue.
5. masonbickers commented on Oct 18, 2026

   

   masonbickers

   on Oct 18, 2026

   More actions

   Additional isolated GET reproduction on macOS arm64, with Firebase CLI 15.34.0 / Firestore emulator 1.22.0, Temurin Java 21.0.12.1, Node 24.19.0, firebase@11.10.0 and @firebase/rules-unit-testing@4.0.1.

   Only a local demo- project, synthetic authentication contexts and two boolean-only user documents are used. All target documents are intentionally absent. There are no production credentials, customer data or application rules in this example. The four predicate forms below each compare a present boolean field on an existing user document; the missing-property case is deliberately broken as an alert control.

   Observed in a fresh local run:

   Predicate	Synthetic denied user (allowed: false)	Synthetic allowed user (allowed: true)
   direct / exists / type / ternary	Denied, each with generic evaluation error ... false for 'get'	GET succeeds (nonexistent target snapshot)
   literal false	Clean denial	Clean denial
   deliberately broken missingFlag	Denied with explicit Property missingFlag is undefined on object.	Same explicit error

   12/12 expected authorization/error-control assertions pass; six SDK responses still contain evaluation diagnostics (four generic, two genuine faults). This is not clean diagnostic acceptance. The separate diagnostic smoke check below exits 1. Our required application diagnostic checks remain unchanged and failing; we are not treating a denial or assertion pass as proof that a diagnostic is harmless. This reproduction does not establish that every application diagnostic has this cause, or make a production-parity claim.

   To reproduce in a new disposable directory, save the following four files. Install the listed versions (the executed run reused an existing installation of those exact versions; a fresh npm install was not separately tested):

   npm init -y
   npm install --save-exact firebase@11.10.0 @firebase/rules-unit-testing@4.0.1 firebase-tools@15.34.0
   npx firebase emulators:exec --only firestore --project demo-firestore-diagnostics --config firebase.json "node --test --test-concurrency=1 reproduce.mjs"
   node strict.mjs

   The last command is expected to exit 1 while these diagnostics remain. No Firebase login is required. Use free local ports as configured below.

   firebase.json:

   {
     "firestore": { "rules": "firestore.rules" },
     "emulators": {
       "firestore": { "host": "127.0.0.1", "port": 18128 },
       "hub": { "host": "127.0.0.1", "port": 18129 },
       "logging": { "host": "127.0.0.1", "port": 18130 },
       "ui": { "enabled": false },
       "singleProjectMode": true
     }
   }

   firestore.rules:

   rules_version = '2';
   service cloud.firestore {
     match /databases/{database}/documents {
       match /direct/{id} {
         allow get: if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.allowed == true;
       }
       match /exists/{id} {
         allow get: if exists(/databases/$(database)/documents/users/$(request.auth.uid)) && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.allowed == true;
       }
       match /type/{id} {
         allow get: if get(/databases/$(database)/documents/users/$(request.auth.uid)).data is map && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.get('allowed', false) == true;
       }
       match /ternary/{id} {
         allow get: if exists(/databases/$(database)/documents/users/$(request.auth.uid)) ? get(/databases/$(database)/documents/users/$(request.auth.uid)).data.get('allowed', false) == true : false;
       }
       match /literal/{id} {
         allow get: if false;
       }
       match /broken/{id} {
         allow get: if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.missingFlag == true;
       }
     }
   }
   

   reproduce.mjs:

   import assert from 'node:assert/strict';
   import fs from 'node:fs';
   import { before, after, test } from 'node:test';
   import { initializeTestEnvironment } from '@firebase/rules-unit-testing';
   import { doc, getDoc, setDoc } from 'firebase/firestore';
   
   const projectId = 'demo-firestore-diagnostics';
   assert.match(process.env.FIRESTORE_EMULATOR_HOST || '', /^(127\.0\.0\.1|localhost):\d+$/);
   let env;
   const records = [];
   before(async () => {
     env = await initializeTestEnvironment({
       projectId,
       firestore: { rules: fs.readFileSync(new URL('./firestore.rules', import.meta.url), 'utf8') },
     });
     await env.clearFirestore();
     await env.withSecurityRulesDisabled(async context => {
       await setDoc(doc(context.firestore(), 'users', 'denied'), { allowed: false });
       await setDoc(doc(context.firestore(), 'users', 'allowed'), { allowed: true });
     });
   });
   after(async () => {
     fs.writeFileSync('results.json', JSON.stringify(records, null, 2));
     await env?.cleanup();
   });
   for (const predicate of ['direct', 'exists', 'type', 'ternary', 'literal', 'broken']) {
     for (const uid of ['denied', 'allowed']) {
       test(`${predicate}: ${uid}`, async () => {
         let error;
         try {
           await getDoc(doc(env.authenticatedContext(uid).firestore(), predicate, 'target'));
         } catch (caught) {
           error = caught;
         }
         const expectedAllow = uid === 'allowed' && !['literal', 'broken'].includes(predicate);
         records.push({ predicate, uid, allowed: !error, code: error?.code ?? null, message: error?.message ?? null });
         assert.equal(!error, expectedAllow);
         if (!expectedAllow) assert.equal(error.code, 'permission-denied');
         if (predicate === 'broken') assert.match(error.message, /Property missingFlag is undefined/);
       });
     }
   }

   strict.mjs:

   import fs from 'node:fs';
   const records = JSON.parse(fs.readFileSync('results.json', 'utf8'));
   const flagged = records.filter(row => /evaluation error|undefined on object/i.test(row.message || ''));
   console.log(`${flagged.length} diagnostic-bearing operations; diagnostics are not waived.`);
   process.exit(flagged.length ? 1 : 0);

   Could a maintainer advise on a supported correction or supported structured diagnostic field that distinguishes an intentional false result from a genuine evaluation exception, while preserving all genuine-error alerts? If this is known behavior, which emulator/CLI version, documented setting or per-request diagnostic field should we test? If it needs a runtime fix, is there a tracking change or corrected build available for validation?

   We are not looking to filter messages, substitute weaker rules or waive negative authorization checks. No such workaround has been applied. The full raw SDK messages are retained in the generated results.json for this synthetic example.