My Portfolio
Research archive
Network Science & Research Infrastructure2026 · Verified scoped dataset release

Vietnam-EU Monthly CN8 Trade Data Pipeline

Poster summary

Python · pandas · Parquet · Eurostat Comext · schema validation

A resumable Eurostat Comext workflow that constructs member-state and EU-total product-level trade tables with explicit schema and quality rules.

01

Research question

How can detailed monthly EU imports from Vietnam be transformed into a versioned CN8 panel without confusing classification, unit, revision, or coverage changes with economic movements?

02

Methodology

Monthly bulk acquisition, resumable Parquet parts, schema validation, CN8 preservation, reporter aggregation, unit-value construction, and documented quality checks

03

Dataset

Verified partial January 2024 release: 15,096 member-state rows and 3,061 EU-total rows

04

Main finding

The current release validates the construction workflow for one month; it does not yet support the planned 2018-to-latest longitudinal coverage or policy-effect analysis.

Vietnam-EU Monthly CN8 Trade Data Pipeline

Overview

This project builds a reproducible pipeline for EU member-state imports originating in Vietnam at monthly CN8 product detail. It retains member-state rows, produces an EU-total aggregation, and documents the reporting grain, variable definitions, raw-data retention, and quality controls.

The verified public artifact is a partial January 2024 release. The repository documentation describes a broader 2018-to-latest target, but that target is not presented as completed.

Research Question

How can detailed monthly EU imports from Vietnam be transformed into a versioned CN8 panel without confusing classification, unit, revision, or coverage changes with economic movements?

Economic or Technical Motivation

Detailed trade releases are revised over time, product codes change across nomenclatures, unit values can be noisy, and missing product-country-month records are not automatically zero trade. These issues can create false entry, exit, price, or diversification signals if they are not made explicit.

Data

The verified member-state Parquet file contains 15,096 rows and 17 columns. The verified EU-total file contains 3,061 rows and 18 columns. Both cover January 2024 only. Unit value is missing in 874 member-state rows and 66 EU-total rows where the necessary quantity information is unavailable; supplementary-unit codes themselves are present.

The main processed grain is reporter by partner by period by CN8 product by supplementary unit. The data dictionary includes dataset schema, pipeline outputs, quality checks, EU27 reporter codes, and sources.

Methodology

The crawler processes one month at a time, validates each output, and resumes from valid Parquet parts. Transformations preserve leading zeroes in CN8 codes, aggregate statistical procedures, retain raw filtered rows, and compute member-state and EU-total outputs separately. The project avoids hard-coding an unverified CBAM mapping.

Main Findings

The current files establish a usable one-month release and an explicit data contract. They do not establish trends, trade-policy effects, product upgrading, or complete 2018-to-latest coverage.

Robustness, Validation, or Model Assessment

Documented checks cover schema, unique grain, reporters, periods, numeric types, product-code preservation, quantities, and aggregation. The current transform test does not collect successfully because of a dynamic-import/dataclass initialization error, so test-suite completion is not claimed.

Tools and Technologies

Python, pandas, Parquet, Eurostat Comext bulk data, JSONL manifests, and spreadsheet data dictionaries are directly demonstrated.

Limitations

Only January 2024 is verified. CN revisions require year-specific concordances, chapter 99 and confidentiality can suppress detail, and EU-total imports are not final-consumption destinations. The failed test collection still requires repair.

Deliverables

  • Monthly acquisition and transformation scripts
  • January 2024 member-state and EU-total Parquet releases
  • Data dictionary and quality rules
  • Download manifest and raw/interim/processed directory contract