SpecFormula AI
ISABackend

The Three Specs

How Feature, API Spec, and Entity Spec Work Together

SpecFormula's backend testing framework is built on three core specifications, which we call "The Three Specs": Feature (.feature test files), API Spec (OpenAPI specifications), and Entity Spec (entity and table specifications). These three are interdependent and evolve together, forming the foundation of declarative BDD testing.

Overview

SpecFilePurpose
Feature File*.featureDefines test scenarios, references API Spec and Entity Spec
API Spec*.yml (OpenAPI 3.0 format)Defines API endpoints, request/response structures
Entity Specentity_to_table_mapping.yml + *.sqlDefines entity-to-table mapping and table structure (DDL)

Why Maintain Three Specs Together?

1. Single Source of Truth

In traditional BDD testing, API structure, database schema, and test code are maintained independently, which can easily lead to inconsistencies:

# Problems with traditional approach
API docs say: POST /users requires email field
Database has: users.email_address column
Test code: request.put("mail", value)  ← Inconsistent across all three

SpecFormula establishes a single source of truth through The Three Specs, with direct mapping between spec definitions and tests:

Entity Spec — Defines entity-to-table mapping and table structure

# entity_to_table_mapping.yml
entity_to_table_mapping:
  - user: users
-- DDL (schema.sql)
CREATE TABLE users (user_id INT PRIMARY KEY, email VARCHAR(255) NOT NULL);

API Spec — Defines API structure

# api-spec.yml
paths:
  /users:
    post:
      summary: create user
      requestBody:
        properties:
          email:
            type: string

Tests directly reference specs, no need to redefine

When (No Actor) create user, call table:       # ← Maps to API Spec (summary: create user)
  | email             |
  | alice@example.com |

Then should exist a user, with table:           # ← Maps to Entity Spec (user → users)
  | email             |
  | alice@example.com |

2. Cascading Changes

When specs change, Lint automatically checks the consistency of Feature, API Spec, and Entity Spec, catching issues during the CI phase:

API Schema Change

API Spec adds required field: requestBody.properties adds department (required)
            ↓
Lint checks .feature: whether all "create user" call tables include the department field

Table Schema Change

DDL column change: users table renames email to email_address
            ↓
Lint checks .feature: whether all "user" with tables use the correct column name

3. Spec-Driven Development

The Three Specs support a "spec first, implementation later" development workflow:

Phase 1: Product Manager defines API Spec
         ↓
Phase 2: DBA designs database tables, creates Entity Spec
         ↓
Phase 3: QA writes .feature tests (tests will fail at this point)
         ↓
Phase 4: Developer implements features to make tests pass

This workflow ensures spec, implementation, and tests are aligned from the start.


API Spec (OpenAPI 3.0)

File Location

API Spec can be split into multiple .yml files, as long as they follow OpenAPI 3.0 format. Path is specified by config.api.resource_path in isa.yml.

src/test/resources/specs/api/
├── api-spec.yml          # Can be split into multiple files
├── auth-api.yml          # e.g., split by module
└── ...

Core Structure

openapi: 3.0.0
info:
  title: My Application API
  version: 1.0.0

paths:
  /punch:
    post:
      summary: punch          # ← Identifier for ApiCall and ResponseValidate instructions
      operationId: punch
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - agentId
                - punchType
              properties:
                agentId:
                  type: integer
                punchType:
                  type: string
                  enum: [IN, OUT]
                punchTime:
                  type: string
                  format: date-time
      responses:
        "200":
          description: Punch successful
          content:
            application/json:
              schema:
                type: object
                properties:
                  recordId:
                    type: integer
                  agentId:
                    type: integer
                  punchType:
                    type: string
                  punchTime:
                    type: string
                  dailyWorkHours:
                    type: number

Mapping with Feature Instructions

ApiCall — Identifies API endpoint via summary, sends HTTP request:

When (UID="$Agent.id") punch, call table:
  | agentId      | punchType | punchTime              |
  | $Agent.id    | IN        | @time("2026-01-27T09:00:00") |

ResponseValidate — Validates response content via summary and status code:

Then punch(200) response, with table:
  | recordId | agentId | punchType |
  | &isNotNull | 1       | IN        |
Gherkin ElementAPI Spec Mapping
punchpaths./punch.post.summary
agentId, punchType, punchTimerequestBody.properties
recordId, punchType etc. response fieldsresponses.200.schema.properties
HTTP Methodpaths./punch.post (inferred from path definition)

Important Rules

  1. summary must be unique: Each operation's summary must be unique across the entire API Spec, otherwise SpecFormula cannot identify it
  2. Complete schema definition: Request/Response schemas must be fully defined for validation
  3. Correct required marking: Required fields must be marked with required, ensuring tests provide all necessary fields

Entity Spec

File Location

Each Data Source directory contains:

  • entity_to_table_mapping.yml — Defines entity name to table mapping
  • *.sql (DDL) — Defines table structure (multiple files supported)

Path is specified by config.data.source[].resource_path in isa.yml.

src/test/resources/specs/data/
└── primary/
    ├── entity_to_table_mapping.yml   # Entity mapping
    ├── agents.sql                    # DDL (any filename, *.sql, multiple files supported)
    └── punch_records.sql

Core Structure

Entity Mapping — Defines the mapping between business entity names and tables:

# entity_to_table_mapping.yml
entity_to_table_mapping:
  - agent: agents
  - punch record: punch_records

DDL — Defines table schema, used by SpecFormula to validate column existence:

-- agents.sql
CREATE TABLE agents (
    agent_id INT PRIMARY KEY,
    name     VARCHAR(100) NOT NULL,
    email    VARCHAR(255) NOT NULL
);

-- punch_records.sql
CREATE TABLE punch_records (
    record_id  INT PRIMARY KEY,
    agent_id   INT NOT NULL,
    punch_type VARCHAR(10),
    punch_time DATETIME
);

Mapping with EntitySetup/EntityValidate Instructions

Given prepare an agent, with table:
  | name    | email               |
  | Manager | manager@company.com |

Then should exist a punch record, with table:
  | agentId | punchType |
  | 1       | IN        |
Gherkin ElementEntity Mapping
agentagents table
punch recordpunch_records table

Column Name Conversion

DataTable uses camelCase, automatically mapped to database snake_case:

Gherkin: agentId    → DB: agent_id
Gherkin: punchType  → DB: punch_type
Gherkin: punchTime  → DB: punch_time

When matching against actual database columns, the framework applies a three-step resolution strategy for each field name:

  • Step 1: exact match
    • First, it looks for the original field name as-is, for example agentId vs agentId.
  • Step 2: camelCase → snake_case
    • If no exact match is found, it converts agentId to agent_id and tries to match again against the DDL / actual data.
  • Step 3: case-insensitive
    • If the first two steps still fail, it finally falls back to case-insensitive comparison, so agent_id, AGENT_ID, and Agent_Id are treated as the same column.

The following table shows some common combinations (left is the Feature/DataTable field, right is the DDL/actual Map key):

Feature field nameDDL / Map field nameMatch?Explanation
agentIdagentId✅ YesStep 1: direct equals match
agentIdagent_id✅ YesStep 2: camelCase → snake_case
agentIdAGENT_ID✅ YesStep 2 to agent_id, then Step 3 equalsIgnoreCase

In practice, as long as you consistently use camelCase in Spec/Gherkin, the engine can reliably resolve typical snake_case or upper-case column names in the DDL, without requiring manual handling of case or underscores in tests or code.


ISA Config (Configuration)

ISA Config is the configuration file for the SpecFormula engine, defining instruction syntax and spec file paths. It is typically set up once at the beginning of a project and rarely needs to change.

File Location

src/test/resources/isa.yml

Core Structure

config:
  api:
    resource_path: specs/api                          # classpath, used by framework at runtime
    project_path: src/test/resources/specs/api        # source path, used by Lint for error reporting
  data:
    source:
      - name: primary
        resource_path: specs/data/primary             # classpath, used by framework at runtime
        project_path: src/test/resources/specs/data/primary  # source path, used by Lint for error reporting
        db_type: mssql

instructions:
  - name: Time control
    format: ^current time is "(?P<time>[^"]+)"$
    instruction_type: time_control

  - name: Data preparation
    format: ^prepare a (?P<entity>[\w\s]+), with table:$
    instruction_type: entity_setup

  - name: API call
    format: ^\((?:No Actor|UID="(?P<userId>\$[\w.]+)")\) (?P<summary>.+?), call table:$
    instruction_type: api_call

  - name: Response validation with API summary and status code
    format: ^(?P<summary>.+?)\((?P<status_code>\d{3})\) response,?\s*with table:$
    instruction_type: response_validate
    data_format: data_table

  - name: Database validation
    format: ^should exist a (?P<entity>[\w\s]+), with table:$
    instruction_type: entity_validate

  - name: Database non-existence validation
    format: ^should not exist a (?P<entity>[\w\s]+), with table:$
    instruction_type: entity_non_existence_validate

Configuration Details

Config ItemDescription
config.api.resource_pathAPI Spec classpath relative path
config.data.source[]Multiple Data Sources supported, each pointing to a set of Entity Spec
config.data.source[].db_typeDatabase type (embedded, mssql, postgresql, mysql); embedded is a framework-chooses-impl alias
instructions[].formatRegular expression for instruction, used to match Gherkin steps
instructions[].instruction_typeInstruction type, determines execution logic

Iteration Workflow for The Three Specs

Adding a New API Endpoint

1. Add path definition in API Spec
2. Write .feature tests
3. Implement the API
4. Run tests to verify

Modifying Table Structure

1. Update DDL (*.sql)
2. If table name changes, update entity_to_table_mapping.yml mapping
3. If column name changes, update column names in .feature files
4. Run Lint to check consistency
5. Run tests to verify

Changing API Behavior

1. Update request/response schema in api-spec.yml
2. Update expected results in related .feature tests
3. Modify API implementation
4. Run tests to verify

Best Practices

1. Keep Summary Readable

# ✅ Good summary: verb + noun, clearly describes the operation
summary: create user
summary: query order list
summary: update product inventory

# ❌ Avoid: too short or too technical
summary: create
summary: POST user
summary: updateInventoryById

2. Use Business Language for Entity Naming

# ✅ Use names familiar to business team
entity_to_table_mapping:
  - member: members
  - order: orders
  - product: products

# ❌ Avoid technical names
entity_to_table_mapping:
  - member_entity: members
  - tbl_order: orders

3. Run Lint Checks Regularly

# Add Lint step in CI
# TODO: Command to be added

Lint checks:

  • Whether summaries in API Spec are unique
  • Whether tables in Entity Spec entity_to_table_mapping exist in DDL
  • Whether fields used in Feature File are defined in Entity Spec and API Spec

4. CI Automated Testing

Integrate tests into CI pipeline to ensure every commit automatically validates The Three Specs consistency.

Java (Maven)

mvn test

C# (Reqnroll / .NET)

dotnet test

On this page