parsed_result Shape ReferenceSingle source of truth for
parse_jobs.parsed_resultJSONB shape across the v1-ssp legacy path (pre Phase 2 deploy in-flight jobs) and the v2-bundle new path (Phase 2 onward).Used by:
- Phase 2 Task 4 —
SspImportConfirmServiceschema_version dispatch +bundle_restore- Phase 2 Task 7 — excel
_write_all_datawire-up- Phase 2 Task 9 — docx confirm wire-up
Investigation scope (read-only):
app/oscal/service/excel_parser/types.py—ParsedExcel.to_dict()app/oscal/service/excel_parser/parser.py+sheet_handlers.py— what each sheet writes into the result dictapp/module_frame/excel_template/sheet_definitions.py— column-level ColumnDef spec (sheet templates v2.6.0)app/oscal/service/ssp_excel_import_app_service.py—_dict_to_parsed_*helpers (lines 1979–2166),_write_all_data(line ~689)app/oscal/service/ssp_docx_import_app_service.py—_parsed_docx_to_dict,_parsed_ssp_to_dict_overlay,_dict_to_parsed_partiesdomain/oscal/parser/ssp_intermediate.py— v3 dataclasses (ParsedComponent,ParsedLeveragedAuthorization,ParsedInventoryItem,ParsedParty,ParsedSystemCharacteristic,ParsedExcelEntityBundle)domain/oscal/parser/docx_intermediate.py— v1 ParsedDocx (matched_controls etc.)
Authoritative writer: ParsedExcel.to_dict() (app/oscal/service/excel_parser/types.py:47-61).
Discriminator: absent or schema_version != "v2-bundle" is treated as v1-ssp legacy by Task 4 dispatcher.
| key | type | source | notes |
|---|---|---|---|
template_version |
str |
excel_parser/types.py:35 | Excel template SemVer (e.g. v2.6.0). Absent on docx side. |
metadata |
dict | None |
Excel 基本資料 sheet / docx _parsed_ssp_to_dict_overlay |
See §1.3 |
system_characteristic |
dict | None |
Excel 受評標的 sheet (v2.2.0+) |
See §1.4 |
parties_org |
list[dict] |
Excel 單位 sheet |
See §1.5 |
parties_person |
list[dict] |
Excel 參與人員 sheet |
See §1.6 |
devices |
list[dict] |
Excel 設備 sheet |
See §1.7 |
info_systems |
list[dict] |
Excel 資訊系統 sheet |
See §1.8 |
leveraged |
list[dict] |
Excel 外部利用服務 sheet |
See §1.9 |
controls_with_aos |
list[dict] |
Excel 控制項與AO sheet (父子 row 結構) |
See §1.10 |
ref_docs |
list[dict] |
Excel 程序書 sheet |
{doc_name, description} |
validation_errors |
list[dict] |
excel_parser ValidationError.to_dict | {sheet, row, field, code, message, blocking} |
Docx-only extra top-level keys (ssp_docx_import_app_service.py:238-340):
| key | type | notes |
|---|---|---|
matched_controls |
list[dict] |
v1 ParsedDocx — {control_id, control_name, score, score_reason, parsed_implementation_description, current_implementation_description, current_objectives_descriptions, catalog_objective_descriptions, objectives:[{objective_key, parsed_description}]} |
unmatched_paragraphs |
list[dict] |
{paragraph_idx, text, context, rule_score, rule_guess_control_id} |
missing_baseline_controls |
list[str] |
control_id list |
predicted_module_frame_controls |
list[str] |
mode=full only |
summary |
dict |
parser-emitted summary stats |
system_metadata |
dict | None |
v1 SspSystemMetadata serialized |
baseline_controls |
list[dict] |
{control_id, control_name, catalog_control_uid, objective_keys, current_implementation_description} |
parties |
list[dict] |
v2 adapter overlay — see §1.6 (docx merges org+person into single list) |
system_characteristics |
dict |
v2 adapter overlay — ParsedSystemCharacteristics asdict |
leveraged_services |
list[dict] |
v2 adapter overlay — ParsedLeveragedService asdict |
information_types |
list[dict] |
v2 adapter overlay — ParsedInformationType asdict |
framework_specific_props |
dict |
adapter pass-through |
adapter_status |
dict[str,str] |
per-section ok / failed / not_run |
adapter_errors |
dict[str,str] |
optional, only on adapter failure |
Both Excel and docx legacy paths fall under v1-ssp from Task 4's perspective. The docx side never wrote
devices/info_systems/leveragedlegacy Excel keys — it skipped straight to v2 adapter overlay (parties,leveraged_services, etc.). Task 4 bundle_restore only needs to handle the Excel v1-ssp shape; docx in-flight legacy jobs already use the v2 overlay keys and will continue using_legacy_confirmfor the docx-specific control diff workflow.
| top-level key | helper | dataclass produced | location |
|---|---|---|---|
devices |
_dict_to_parsed_devices |
ParsedDevice |
ssp_excel_import_app_service.py:1979 |
info_systems |
_dict_to_parsed_info_systems |
ParsedInformationSystem |
line 1996 |
leveraged |
_dict_to_parsed_leveraged |
ParsedLeveraged |
line 2016 |
system_characteristic |
_dict_to_parsed_system_characteristic |
ParsedSystemCharacteristic |
line 2055 |
controls_with_aos |
_dict_to_parsed_controls |
ParsedControl + nested ParsedAssessmentObjective |
line 2084 |
parties_org + parties_person |
_dict_to_parsed_parties |
ParsedParty |
line 2121 |
metadata schema (Excel 基本資料 sheet)Vertical layout single row. Keys from SHEET_METADATA columns (sheet_definitions.py:82-97):
| key | type | required | notes |
|---|---|---|---|
name |
str |
yes | MF name |
group |
str |
yes | MF group |
version |
str |
yes | MF version |
description |
str | None |
no | |
framework_version_uid |
str | None |
no | hidden col, internal UID |
profile_uid |
str | None |
no | hidden col, internal UID |
frequency |
str |
yes | |
provider |
str | None |
no |
system_characteristic schema (Excel 受評標的 sheet, v2.2.0+)Vertical layout single row (SHEET_SYSTEM_CHARACTERISTIC columns, sheet_definitions.py:266-300). Maps to ParsedSystemCharacteristic.
| key | type | required | notes |
|---|---|---|---|
system_name |
str |
yes | |
security_sensitivity_level |
str | None |
no | enum low / moderate / high |
status |
str | None |
no | enum active / under-development / disposition / other |
target_type |
str | None |
no | |
scope_description |
str | None |
no | |
owner_login_name |
str | None |
no | lookup users (login_name) |
parties_org[i] schema (Excel 單位 sheet)SHEET_ORGS columns (sheet_definitions.py:100-121):
| key | type | required | notes |
|---|---|---|---|
name |
str |
yes | |
short_name |
str | None |
no | |
matched_org_unit |
str | None |
no | lookup org_units — reconciler stage 0 input |
address |
str | None |
no | v2.6.0 |
telephone |
str | None |
no | v2.6.0 |
parent_org |
str | None |
no | legacy v1.x — _dict_to_parsed_parties reads as fallback to matched_org_unit |
role |
str | None |
no | legacy v1.x — _dict_to_parsed_parties defaults to responsible-organization when missing (v2.0 sheet dropped role col) |
email |
str | None |
no | legacy v1.x — _dict_to_parsed_parties reads as email_address |
parties_person[i] schema (Excel 參與人員 sheet)SHEET_PERSONS columns (sheet_definitions.py:124-153):
| key | type | required | notes |
|---|---|---|---|
email |
str |
yes | reconciler stage 0 input |
name |
str |
yes | |
matched_user |
str | None |
no | lookup users — reconciler stage 0 input |
role |
str |
yes | enum manager / reviewer / auditor / viewer |
title |
str | None |
no | v2.6.0 |
telephone |
str | None |
no | v2.6.0 (mapped to ParsedParty.telephone_number) |
address |
str | None |
no | v2.6.0 |
org_unit |
str | None |
no | legacy v1.x — _dict_to_parsed_parties reads as matched_org_unit_label |
Docx side
parties[i](merged list, top-level not split into org/person) isParsedPartyasdict — see §2.5 below for the merged shape.
devices[i] schema (Excel 設備 sheet)SHEET_DEVICES columns (sheet_definitions.py:156-171) + legacy v1.x dropped columns still consumed by _dict_to_parsed_devices:
| key | type | required | notes |
|---|---|---|---|
name |
str |
yes | |
ip |
str |
yes | |
matched_device |
str | None |
no | lookup devices — reconciler stage 0 input |
os |
str | None |
no | legacy v1.x (dropped from v2.0 sheet, still read defensively) |
device_type |
str | None |
no | legacy v1.x |
status |
str | None |
no | legacy v1.x |
purpose |
str | None |
no | legacy v1.x |
info_systems[i] schema (Excel 資訊系統 sheet)SHEET_INFO_SYSTEMS columns (sheet_definitions.py:174-192) + legacy v1.x dropped columns still consumed by _dict_to_parsed_info_systems:
| key | type | required | notes |
|---|---|---|---|
name |
str |
yes | |
abbreviation |
str | None |
no | |
matched_info_system |
str | None |
no | lookup info_systems — reconciler stage 0 input |
description |
str | None |
no | legacy v1.x |
component_type |
str | None |
no | legacy v1.x |
status |
str | None |
no | legacy v1.x |
system_owner |
str | None |
no | legacy v1.x (mapped to ParsedInformationSystem.system_owner_label) |
leveraged[i] schema (Excel 外部利用服務 sheet)SHEET_LEVERAGED columns (sheet_definitions.py:195-252). _dict_to_parsed_leveraged skips rows where both service_name and provider are blank.
| key | type | required | notes |
|---|---|---|---|
category |
str | None |
no | enum external-service / interconnection / api / cli / leveraged-authorization / other |
service_name |
str |
yes | |
provider |
str | None |
no | not required as of v2.4.0 |
date_authorized |
str / date / datetime / None |
no | ISO date string after to_dict(); helper coerces back to date |
status |
str | None |
no | enum operational / under-development / disposition / other |
purpose |
str | None |
no | |
description |
str | None |
no | v2.4.0 |
protocol |
str | None |
no | v2.5.0 |
security_auth |
str | None |
no | v2.5.0 |
nature_of_agreement |
str | None |
no | v2.5.0 |
impact_level |
str | None |
no | enum low / moderate / high / li-saas |
data_types |
str | None |
no | |
authorized_users |
str | None |
no | |
fedramp_package_id |
str | None |
no | |
party |
str | None |
no | legacy v1.x (mapped to ParsedLeveraged.party_label) |
controls_with_aos[i] schema (Excel 控制項與AO sheet)Parent-child row structure (sheet_handlers.py:194-238). Parent row (statement_id is None) contains nested objectives list of child rows (statement_id is the AO uid string).
Parent row:
| key | type | required | notes |
|---|---|---|---|
statement_id |
None |
— | parent marker (always None for parent) |
control_id |
str |
yes | |
control_name |
str | None |
no | |
objective_id |
str | None |
no | |
objective_name |
str | None |
no | |
impl_status |
str |
yes | enum implemented / partial / planned / alternative / not_applicable |
statement |
str |
yes | |
reference_doc |
str | None |
no | lookup ref_docs_mf |
include_in_profile |
str | None |
no | enum TRUE / FALSE (v1.2.0+) |
_target_in_profile |
bool |
yes | parser-computed from include_in_profile (default True for legacy templates) |
objectives |
list[dict] |
yes | child rows (may be empty) |
Child row (within objectives[]):
| key | type | required | notes |
|---|---|---|---|
statement_id |
str |
yes | AO uid (e.g. AC-1.a.1) |
control_id |
str |
yes | matches parent |
objective_id |
str | None |
no | |
objective_name |
str | None |
no | |
impl_status |
str | None |
no | |
statement |
str | None |
no | |
reference_doc |
str | None |
no |
Phase 2 does not change the controls path —
controls_with_aossticks with v1 shape in v2-bundle.
Authoritative writer: Phase 2 Task 7 (excel) + Task 9 (docx). Both pipelines output ParsedExcelEntityBundle v3 dataclasses serialized via asdict()-equivalent shape, then dispatched by Task 4 confirm_service on schema_version.
| key | type | notes |
|---|---|---|
schema_version |
str |
discriminator. "v2-bundle" for new path. |
template_version |
str |
Excel template SemVer (carried through for FE display) |
components |
list[dict] |
aligned to ParsedComponent — see §2.2 |
leveraged_authorizations |
list[dict] |
aligned to ParsedLeveragedAuthorization — see §2.3 |
inventory_items |
list[dict] |
aligned to ParsedInventoryItem — see §2.4 |
parties |
list[dict] |
merged from parties_org + parties_person — see §2.5 |
controls_with_aos |
list[dict] |
unchanged from v1 shape — see §1.10 |
metadata |
dict | None |
unchanged from v1 shape — see §1.3 |
system_characteristic |
dict | None |
unchanged from v1 shape — see §1.4 |
ref_docs |
list[dict] |
unchanged from v1 shape — {doc_name, description} |
import_warnings |
list[dict] |
new — see §2.6 |
validation_errors |
list[dict] |
unchanged from v1 shape |
components[i] schemaAligned to ParsedComponent (domain/oscal/parser/ssp_intermediate.py:222-249).
| key | type | dataclass field | notes |
|---|---|---|---|
title |
str |
.title |
required |
component_type |
str |
.component_type |
OSCAL 14 enum + other — this-system / system / interconnection / software / hardware / service / policy / physical / process-procedure / plan / guidance / standard / validation / network / other |
description |
str | None |
.description |
|
purpose |
str | None |
.purpose |
|
status |
str | None |
.status |
enum operational / under-development / under-major-modification / disposition / other |
leveraged_authorization_ref |
str | None |
.leveraged_authorization_ref |
LA title string at parse time. Task 3 normalizer resolves to a real LA in the bundle (sets to None + emits warning if no match); reconciler later resolves to LA uid. |
protocol |
str | None |
.protocol |
framework-specific prop |
port_ranges |
str | None |
.port_ranges |
framework-specific prop |
security_auth |
str | None |
.security_auth |
framework-specific prop |
matched_party_uuid |
str | None |
.matched_party_uuid |
filled by reconciler |
match_method |
str |
.match_method |
MatchMethod enum value as string (e.g. "unmatched" / "exact" / "normalized" / "fuzzy_high" / "fuzzy_low") |
match_confidence |
float |
.match_confidence |
0.0–1.0 |
leveraged_authorizations[i] schemaAligned to ParsedLeveragedAuthorization (ssp_intermediate.py:252-268).
| key | type | dataclass field | notes |
|---|---|---|---|
title |
str |
.title |
required. dedup key for Task 3 normalizer. |
provider |
str | None |
.provider |
CSP / vendor name → party (CSP) at reconcile time |
date_authorized |
str (ISO) | None |
.date_authorized |
date serialized to ISO string in JSONB |
fedramp_package_id |
str | None |
.fedramp_package_id |
|
impact_level |
str | None |
.impact_level |
enum low / moderate / high / li-saas |
data_types |
str | None |
.data_types |
|
nature_of_agreement |
str | None |
.nature_of_agreement |
|
authorized_users |
str | None |
.authorized_users |
|
remarks |
str | None |
.remarks |
inventory_items[i] schemaAligned to ParsedInventoryItem (ssp_intermediate.py:271-290).
| key | type | dataclass field | notes |
|---|---|---|---|
description |
str |
.description |
OSCAL-required (NOT NULL) |
asset_id |
str | None |
.asset_id |
|
asset_tag |
str | None |
.asset_tag |
|
ipv4_address |
str | None |
.ipv4_address |
|
mac_address |
str | None |
.mac_address |
|
fqdn |
str | None |
.fqdn |
|
hostname |
str | None |
.hostname |
|
software_name |
str | None |
.software_name |
|
os_name |
str | None |
.os_name |
|
implemented_component_refs |
list[str] |
.implemented_component_refs |
component title strings at parse time. Task 3 normalizer prunes unresolved refs (emits warning); reconciler later resolves to component uids. |
parties[i] schemaAligned to ParsedParty (ssp_intermediate.py:52-81). party_type discriminates person vs organization in a single list (replacing v1's parties_org / parties_person split).
| key | type | dataclass field | notes |
|---|---|---|---|
name |
str |
.name |
required |
party_type |
str |
.party_type |
"person" or "organization" |
role |
str | None |
.role |
e.g. system-owner / security-officer / responsible-organization |
title |
str | None |
.title |
OSCAL Party title (for person) |
email_address |
str | None |
.email_address |
|
telephone_number |
str | None |
.telephone_number |
|
address |
str | None |
.address |
|
short_name |
str | None |
.short_name |
(organization only) |
remarks |
str | None |
.remarks |
|
matched_user_id |
int | None |
.matched_user_id |
reconciler-filled (public.members.id) |
matched_org_unit_id |
int | None |
.matched_org_unit_id |
reconciler-filled (public.org_units.id) |
target_party_uid |
str | None |
.target_party_uid |
diff-service paired current party uid (write strategy precision target) |
match_method |
str |
.match_method |
MatchMethod enum value as string |
match_confidence |
float |
.match_confidence |
0.0–1.0 |
matched_user_label |
str | None |
.matched_user_label |
for FE preview UI |
matched_org_unit_label |
str | None |
.matched_org_unit_label |
for FE preview UI |
import_warnings[i] schemaPer Phase 2 Task 2 ImportWarning dataclass (domain/oscal/import_pipeline/warnings.py, planned).
| key | type | notes |
|---|---|---|
ref |
str |
the offending value (e.g. LA title not found) |
reason |
str |
warning message (e.g. "leveraged_authorization not found in bundle") |
context |
dict |
additional context for FE display (e.g. {"component_title": "AcmeCorp Backup Service"}) |
source_table |
str | None |
optional — "components" / "leveraged_authorizations" / "inventory_items" |
Emitted by Task 3 normalizer when:
(title, component_type) collapsed → source_table="components"title collapsed → source_table="leveraged_authorizations"ParsedComponent.leveraged_authorization_ref not found in bundle → set to NoneParsedInventoryItem.implemented_component_refs entry not found → prunedbundle_restore reference)Note: Phase 2 does NOT auto-migrate legacy v1-ssp
parsed_resultto v2-bundle.SspImportConfirmServiceraisesLegacyConfirmRequiredwhen it detects v1-ssp (noschema_versionorschema_version != "v2-bundle"), and the caller dispatches to the existing_legacy_confirmpath. This conversion table is informational reference only — useful for:
- Future FE migration script (offer user to discard legacy parse jobs)
- Retrospective debugging when comparing v1 vs v2 entity counts
- Hand-written test fixtures translating v1 dicts to v2 dataclasses
| v1 location | v2 location | mapping |
|---|---|---|
devices[i] |
components[j] |
component_type='hardware', title=devices[i].name, description=devices[i].purpose (legacy field), port_ranges=None (no v1 source), other framework fields → None |
info_systems[i] |
components[j] |
component_type='system', title=info_systems[i].name, description=info_systems[i].description (legacy field), purpose=None |
leveraged[i] (Component side) |
components[j] |
component_type='service', title=leveraged[i].service_name, description=leveraged[i].description, purpose=leveraged[i].purpose, status=leveraged[i].status, protocol=leveraged[i].protocol, security_auth=leveraged[i].security_auth, leveraged_authorization_ref=leveraged[i].service_name (1:1 pairing) |
leveraged[i] (LA side) |
leveraged_authorizations[k] |
title=leveraged[i].service_name, provider=leveraged[i].provider, date_authorized=leveraged[i].date_authorized, fedramp_package_id=leveraged[i].fedramp_package_id, impact_level=leveraged[i].impact_level, data_types=leveraged[i].data_types, nature_of_agreement=leveraged[i].nature_of_agreement, authorized_users=leveraged[i].authorized_users |
parties_org[i] |
parties[j] |
party_type='organization', plus all OSCAL Party fields (name / short_name / email_address / address / telephone_number / matched_org_unit_label); role defaults to 'responsible-organization' if missing |
parties_person[i] |
parties[j] |
party_type='person', plus all OSCAL Party fields (name / email_address / role / title / telephone_number / address / matched_user_label); name falls back to email when blank |
parties_org[i].parent_org (legacy) |
parties[j].matched_org_unit_label |
fallback when matched_org_unit absent (pre-v2.0.1 templates) |
controls_with_aos |
controls_with_aos |
unchanged — copied as-is |
metadata |
metadata |
unchanged — copied as-is |
system_characteristic |
system_characteristic |
unchanged — copied as-is |
ref_docs |
ref_docs |
unchanged — copied as-is |
| n/a | inventory_items |
no v1 source — v2-bundle inventory_items requires Excel 04_設備 → Component split where v1 conflated device-as-inventory + device-as-component into single devices rows. Migration would have to leave inventory_items=[] and let user re-upload. |
| n/a | import_warnings |
no v1 source — initialized [] for converted bundle |
| n/a | schema_version |
set to "v2-bundle" |
devices rows conflate "physical inventory" (ip / mac / hostname) with "component" (name / purpose / status). v2 splits these into separate inventory_items and components records — but a v1 row has no signal for "is this a physical asset I want tracked separately from the abstract component?". A retrofit must default to "component only" and lose the inventory side.leveraged[i].service_name is reused as both the Component title and the LA title (single field). If a user accidentally created two leveraged rows with the same service_name but different provider, the v1 path used (service_name, provider) as the natural key while v2 normalizer dedups LAs by title alone — converter must surface a warning when collapsing.parties_org v2.x sheet dropped email / role columns. Existing in-flight jobs that have email filled in (from v1.x templates still defensive-read by _dict_to_parsed_parties) would round-trip cleanly, but the converter must read the legacy fields, not the stripped v2.x set.devices / info_systems / leveraged keys — they went straight to v2 adapter overlay (parties, leveraged_services, information_types). bundle_restore on docx in-flight jobs would need a separate path (out of Phase 2 scope; Task 9 handles docx parse v3 directly).template_version carry-through: v1-ssp Excel writes template_version at top level (ParsedExcel.to_dict()). v2-bundle plan does not explicitly list it. Recommendation: keep it in v2-bundle top-level for FE display continuity. Added to §2.1 table.
Docx side has no Excel-equivalent legacy keys: docx in-flight jobs use a different key set (matched_controls, parties, leveraged_services, etc.). Task 4 dispatcher logic should:
schema_version == "v2-bundle" → v2 pathcontrols_with_aos parent/child structure not v3-aligned: Phase 2 plan intentionally keeps v1 shape (parent dict with nested objectives[]). The ParsedControl / ParsedAssessmentObjective dataclasses are unchanged from v2.x. This is correct per the plan ("Phase 2 not changing control path").
MatchMethod serialization: dataclass field is MatchMethod enum (StrEnum); asdict() serializes to the string value. v2-bundle JSONB stores string, deserialization in bundle_restore must map back to enum (or callers accept str).
date_authorized serialization: ParsedLeveragedAuthorization.date_authorized is date, JSONB stores ISO string. bundle_restore must date.fromisoformat() when reading back. Same pattern as legacy _dict_to_parsed_leveraged (line 2026-2036).
role mandatory in parties_person legacy schema but free-form in v2-bundle?: v1 Excel SHEET_PERSONS.role enum is manager / reviewer / auditor / viewer. v3 ParsedParty.role is Optional[str] with OSCAL-style examples (system-owner / security-officer). Two different domains — v1 was project-participant role, v2 should be OSCAL responsible-party role. Open question for Task 7 (excel wire-up): does the new Excel template emit OSCAL-style roles or keep project-participant roles? Implicit answer in current code: kept as-is, role mapping done at write-strategy layer.
ref_docs standalone: v2-bundle plan in implementation-plan-phase2.md §Task 1 sketch did not list ref_docs. Recommendation: keep ref_docs as top-level key in v2-bundle — added to §2.1 table — since it's a separate orthogonal concern (template ref doc list, not a v3 entity refactor target).