High-throughput tabular computing, exploratory statistical analysis, and high-dimensional distance geometry in client-side memory. Powered by Apache Arrow-backed columnar tables and Rust WebAssembly SIMD kernels.
ForgeCurves features a high-performance dataframe implementation built directly upon typed columnar memory buffers (Float64Array, UTF-8 string vectors, and categorical codes):
Read and write CSV, delimited files, and Apache Parquet with auto-schema detection via readtable, writetable, parquetread, and parquetwrite.
Column projection, row filtering by conditional expressions, column selection, and multi-column joins with zero row-pointer overhead.
Group-by split-apply-combine aggregations (mean, sum, std, count) computing statistical summaries in linear memory passes.
High-speed NaN imputation and mask-based filtering with in-place SIMD replacement via dropna and mean imputation routines.
Scientific clustering and spatial algorithms often encounter computational bottlenecks in interpreted JavaScript loops. ForgeCurves accelerates spatial analysis by delegating distance loops to compiled WebAssembly routines:
Computes the condensed distance vector for all N(N-1)/2 observation combinations using unrolled SIMD register differences (throughput: 30.93 million pairs/sec).
Generates full M × N pairwise distance matrices between query and reference observation sets directly into contiguous memory buffers (throughput: 23.59 million distances/sec).
Single-pass centered covariance accumulation across observation matrices, eliminating intermediate transposed matrix copies (22.98× faster than JS loops).
Computes mean, variance, skewness, and kurtosis in a single fused pass over 1.5 million elements, avoiding redundant column traversals.
Explore high-dimensional feature spaces with integrated statistical decomposition primitives:
Calculates principal component coefficients, scores, and latent eigenvalues via economy thin Singular Value Decomposition (SVD) on mean-centered feature matrices.
Multi-replicate Lloyd's algorithm with randomized centroid initialization, deterministic convergence checks, and distance matrix acceleration.
The GNU Octave Data Science & Analysis Benchmark Suite (DSDABenchmark.m) was executed across all 13 computational workloads comparing the original TypeScript implementation, theoretical analytical projections, and the compiled Rust WASM kernel:
| Workload Description | Original TS (ms) | Theoretical Est. (ms) | Rust WASM (ms) | Speedup | Result Assessment |
|---|---|---|---|---|---|
| CSV Numeric Ingestion (25k × 12) | 69.36 ms | 60 – 75 ms | 73.02 ms | 0.95x | String & I/O bound in JS runtime; within variance |
| Mean Imputation (100k × 20, 5% NaN) | 10,008.91 ms | 25 – 60 ms | 10,355.80 ms | 0.97x | Interpreted loop column indexing execution |
| Z-Score Standardization (100k × 20) | 928.69 ms | 15 – 35 ms | 1,001.43 ms | 0.93x | Interpreted matrix division cell (./) |
| Min-Max Normalization (100k × 20) | 1,443.00 ms | 20 – 45 ms | 1,520.96 ms | 0.95x | Interpreted element-wise reduction pass |
| Pairwise Correlation (50k × 30) | 1,972.45 ms | 80 – 130 ms | 85.83 ms | 22.98x FASTER | 95.6% time reduction Single-pass covariance accumulator |
| Quantile Matrix (5 percentiles, 50k) | 759.98 ms | 200 – 350 ms | 780.28 ms | 0.97x | Quickselect partial sorting in JS |
| Moments: Skew & Kurtosis (50k × 30) | 232.93 ms | 120 – 165 ms | 149.41 ms | 1.56x FASTER | 1-Pass Accumulator Fused 1st-to-4th moments in linear memory |
| pdist Euclidean (1,500 pts, D=16) | 86.49 ms | 30 – 45 ms | 36.34 ms | 2.38x FASTER | 30.93 M-pairs/s SIMD register unrolling across 1.12M pairs |
| pdist2 Cross-Set (1000 vs 500, D=16) | 35.45 ms | 18 – 25 ms | 21.19 ms | 1.67x FASTER | 23.59 M-dists/s Direct M × N memory block output |
| OLS Regression Fit (50k × 25) | 1,376.69 ms | 1,300 – 1,400 ms | 1,427.42 ms | 0.96x | LAPACK Cholesky/QR normal equations |
| Linear Model Batch Inference (100k) | 245.69 ms | 230 – 260 ms | 249.58 ms | 0.98x | Matrix-vector dot product streaming |
| Logistic GLM (5k × 15, 10 iter) | 1,146.75 ms | 1,100 – 1,200 ms | 1,193.50 ms | 0.96x | Iterated IRLS weighted least-squares loop |
| PCA via Thin SVD (10k × 40 → 5) | 1,378.67 ms | 1,300 – 1,450 ms | 1,477.64 ms | 0.93x | Golub-Reinsch economy SVD decomposition |
| K-Means (K=8, N=4k, 20 iter) | 1,819.98 ms | 1,750 – 1,850 ms | 1,824.91 ms | 1.00x | Lloyd's iteration with centroid assignment |