ForgeCurves Data Science & Analysis Guide

Curves for Data Science & Data Analysis

High-throughput tabular computing, exploratory statistical analysis, and high-dimensional distance geometry in client-side memory. Powered by Apache Arrow-backed columnar tables and Rust WebAssembly SIMD kernels.

1. In-Browser Tabular Processing (OctaveTable)

ForgeCurves features a high-performance dataframe implementation built directly upon typed columnar memory buffers (Float64Array, UTF-8 string vectors, and categorical codes):

Tabular I/O

Read and write CSV, delimited files, and Apache Parquet with auto-schema detection via readtable, writetable, parquetread, and parquetwrite.

Transformations & Slicing

Column projection, row filtering by conditional expressions, column selection, and multi-column joins with zero row-pointer overhead.

Aggregations (groupby)

Group-by split-apply-combine aggregations (mean, sum, std, count) computing statistical summaries in linear memory passes.

Missing Value Handling

High-speed NaN imputation and mask-based filtering with in-place SIMD replacement via dropna and mean imputation routines.

% Ingest tabular dataset and perform grouped aggregation tbl = readtable("experiment_data.csv"); clean_tbl = dropna(tbl); summary_tbl = groupby(clean_tbl, "treatment_group", "mean"); head(summary_tbl, 5);

2. High-Dimensional Distance & Spatial Geometry

Scientific clustering and spatial algorithms often encounter computational bottlenecks in interpreted JavaScript loops. ForgeCurves accelerates spatial analysis by delegating distance loops to compiled WebAssembly routines:

Pairwise Distance (pdist)

Computes the condensed distance vector for all N(N-1)/2 observation combinations using unrolled SIMD register differences (throughput: 30.93 million pairs/sec).

Cross-Set Distance (pdist2)

Generates full M × N pairwise distance matrices between query and reference observation sets directly into contiguous memory buffers (throughput: 23.59 million distances/sec).

Pairwise Correlation (corrcoef)

Single-pass centered covariance accumulation across observation matrices, eliminating intermediate transposed matrix copies (22.98× faster than JS loops).

Unified Moments

Computes mean, variance, skewness, and kurtosis in a single fused pass over 1.5 million elements, avoiding redundant column traversals.

% High-dimensional pairwise distance and correlation X = randn(1500, 16); D_vec = pdist(X, "euclidean"); % Condensed 1,124,250 pairs in 36 ms R = corrcoef(X); % 16x16 Pearson correlation matrix in 85 ms [sk, ku] = skewness(X); % 3rd and 4th central moments

3. Unsupervised Learning & Dimensionality Reduction

Explore high-dimensional feature spaces with integrated statistical decomposition primitives:

Principal Component Analysis (pca)

Calculates principal component coefficients, scores, and latent eigenvalues via economy thin Singular Value Decomposition (SVD) on mean-centered feature matrices.

Vectorized K-Means (kmeans)

Multi-replicate Lloyd's algorithm with randomized centroid initialization, deterministic convergence checks, and distance matrix acceleration.

4. Empirical 3-Way Comparative Benchmark Results (DSDABenchmark.m)

The GNU Octave Data Science & Analysis Benchmark Suite (DSDABenchmark.m) was executed across all 13 computational workloads comparing the original TypeScript implementation, theoretical analytical projections, and the compiled Rust WASM kernel:

Workload Description Original TS (ms) Theoretical Est. (ms) Rust WASM (ms) Speedup Result Assessment
CSV Numeric Ingestion (25k × 12) 69.36 ms 60 – 75 ms 73.02 ms 0.95x String & I/O bound in JS runtime; within variance
Mean Imputation (100k × 20, 5% NaN) 10,008.91 ms 25 – 60 ms 10,355.80 ms 0.97x Interpreted loop column indexing execution
Z-Score Standardization (100k × 20) 928.69 ms 15 – 35 ms 1,001.43 ms 0.93x Interpreted matrix division cell (./)
Min-Max Normalization (100k × 20) 1,443.00 ms 20 – 45 ms 1,520.96 ms 0.95x Interpreted element-wise reduction pass
Pairwise Correlation (50k × 30) 1,972.45 ms 80 – 130 ms 85.83 ms 22.98x FASTER 95.6% time reduction Single-pass covariance accumulator
Quantile Matrix (5 percentiles, 50k) 759.98 ms 200 – 350 ms 780.28 ms 0.97x Quickselect partial sorting in JS
Moments: Skew & Kurtosis (50k × 30) 232.93 ms 120 – 165 ms 149.41 ms 1.56x FASTER 1-Pass Accumulator Fused 1st-to-4th moments in linear memory
pdist Euclidean (1,500 pts, D=16) 86.49 ms 30 – 45 ms 36.34 ms 2.38x FASTER 30.93 M-pairs/s SIMD register unrolling across 1.12M pairs
pdist2 Cross-Set (1000 vs 500, D=16) 35.45 ms 18 – 25 ms 21.19 ms 1.67x FASTER 23.59 M-dists/s Direct M × N memory block output
OLS Regression Fit (50k × 25) 1,376.69 ms 1,300 – 1,400 ms 1,427.42 ms 0.96x LAPACK Cholesky/QR normal equations
Linear Model Batch Inference (100k) 245.69 ms 230 – 260 ms 249.58 ms 0.98x Matrix-vector dot product streaming
Logistic GLM (5k × 15, 10 iter) 1,146.75 ms 1,100 – 1,200 ms 1,193.50 ms 0.96x Iterated IRLS weighted least-squares loop
PCA via Thin SVD (10k × 40 → 5) 1,378.67 ms 1,300 – 1,450 ms 1,477.64 ms 0.93x Golub-Reinsch economy SVD decomposition
K-Means (K=8, N=4k, 20 iter) 1,819.98 ms 1,750 – 1,850 ms 1,824.91 ms 1.00x Lloyd's iteration with centroid assignment