Introduction

Official repository of "FastEnhancer: Speed-Optimized Streaming Neural Speech Enhancement" (accepted to ICASSP 2026).
Paper | Documentation

Install

Please refer to document.

Datasets

Please refer to document.

Training

Please refer to document.

Inference

PyTorch Inference

Pytorch checkpoints and tensorboard logs are provided in releases.
Please refer to document for calculating objective metrics.
Please refer to document for pytorch inference.

ONNXRuntime Inference

ONNX models are provided in releases.
Please refer to document for streaming inference using ONNXRuntime.

Results

Voicebank-Demand 16kHz

• Except for GTCRN, we trained each model five times with five different seed and report the average scores.

Table 1. Performance on Voicebank-Demand testset.

Model	Para. (K)	MACs	RTF (Xeon)	RTF (M1)	RTF (M5)	DNSMOS (P.808)	DNSMOS (P.835)			SCOREQ	SISDR	PESQ	STOI	ESTOI	WER
							SIG	BAK	OVL
GTCRNa	24	40M	0.060	0.042	0.0264	3.43	3.36	4.02	3.08	0.330	18.8	2.87	0.940	0.848	3.6
LiSenNetb	37	56M	-	-	-	3.34	3.30	3.90	2.98	0.425	13.5	3.08	0.938	0.842	3.7
LiSenNetc	37	56M	0.034	0.028	0.0172	3.42	3.34	4.03	3.07	0.335	18.5	2.98	0.941	0.851	3.4
FSPENd	79	64M	0.046	0.038	0.0244	3.40	3.33	4.00	3.05	0.324	18.4	3.00	0.942	0.850	3.6
BSRNNd	334	245M	0.059	0.062	0.0307	3.44	3.36	4.00	3.07	0.303	18.9	3.06	0.942	0.855	3.4
FastEnhancer_B	91	262M	0.022	0.026	0.0110	3.47	3.38	4.02	3.10	0.285	19.0	3.13	0.945	0.861	3.2
FastEnhancer_T	22	60M	0.012	0.013	0.0058	3.42	3.34	4.01	3.06	0.334	18.6	2.99	0.940	0.850	3.6
FastEnhancer_B	91	262M	0.022	0.026	0.0110	3.47	3.38	4.02	3.10	0.285	19.0	3.13	0.945	0.861	3.2
FastEnhancer_S	194	664M	0.034	0.048	0.0189	3.49	3.40	4.03	3.12	0.265	19.2	3.19	0.947	0.866	3.2
FastEnhancer_M	492	2.9G	0.101	0.173	0.0386	3.48	3.39	4.02	3.11	0.243	19.4	3.24	0.950	0.873	2.8
FastEnhancer_L	1105	12G	0.313	0.632	0.1052	3.53	3.44	4.04	3.16	0.239	19.6	3.26	0.952	0.877	3.1

^a Evaluated using the official checkpoint.
^b Trained using the official training code. Not streamable because of input normalization and griffin-lim. Thus, RTFs are not reported.
^c To make the model streamable, input normalization and griffin-lim are removed. Trained following the experimental setup of FastEnhancer (same loss function, same optimizer, etc. Only differences are the model architectures).
^d Re-implemented and trained following the experimental setup of FastEnhancer (same loss function, same optimizer, etc. Only differences are the model architectures).

DNS-Challenge 16kHz

• Trained using DNS-Challenge-3 wideband training dataset.
  • Without emotional_speech and singing_voice.
  • With VCTK-0.92 clean speech except p232 and p257 speakers.
  • RIRs were not convolved to the clean speech.
  • Unlike in Voicebank-Demand, we didn't use PESQLoss.
• Tested using DNS-Challenge-1 dev-testset-synthetic-no-reverb dataset.
• We trained each model only once with one random seed.

Table 2. Performance on DNS-Challenge1 dev-testset-synthetic-no-reverb.

Model	Para. (K)	MACs	RTF (Xeon)	RTF (M1)	RTF (M5)	DNSMOS (P.808)	DNSMOS (P.835)			SCOREQ	SISDR	PESQ	STOI	ESTOI
							SIG	BAK	OVL
GTCRNa	24	40M	0.060	0.042	0.0264	3.85	3.35	3.98	3.05	0.551	14.8	2.26	0.934	0.871
LiSenNetb	37	56M	0.034	0.028	0.0172	3.82	3.39	4.08	3.14	0.487	16.3	2.58	0.947	0.893
FSPENb	79	64M	0.046	0.038	0.0244	3.82	3.37	4.09	3.13	0.510	15.8	2.43	0.943	0.885
BSRNNb	334	245M	0.059	0.062	0.0307	3.89	3.41	4.11	3.18	0.441	16.7	2.61	0.951	0.901
FastEnhancer_B	91	262M	0.022	0.026	0.0110	3.92	3.43	4.12	3.20	0.396	16.7	2.69	0.953	0.903
FastEnhancer_T	22	60M	0.012	0.013	0.0058	3.81	3.35	4.07	3.10	0.522	15.4	2.43	0.940	0.879
FastEnhancer_B	91	262M	0.022	0.026	0.0110	3.92	3.43	4.12	3.20	0.396	16.7	2.69	0.953	0.903
FastEnhancer_S	194	664M	0.034	0.048	0.0189	3.96	3.46	4.13	3.23	0.373	17.5	2.79	0.960	0.914
FastEnhancer_M	492	2.9G	0.101	0.173	0.0386	3.98	3.48	4.14	3.26	0.345	18.4	2.78	0.965	0.924
FastEnhancer_L	1105	12G	0.313	0.632	0.1052	4.02	3.51	4.16	3.29	0.298	19.5	2.94	0.971	0.935

^a Evaluated using the official checkpoint. It should be noted that this model was trained for both noise suppression and de-reverberation, whereas FastEnhancers were trained only for noise suppression. If GTCRN is trained for noise suppression only, its performance may be higher.
^b Re-implemented and trained following the experimental setup of FastEnhancer (same loss function, same optimizer, etc. Only differences are the model architectures).

48kHz

• We tried to include only high-quality, truly full-band speech & noise datasets (Table 3).
• We trained each model only once with one random seed.
• We observed that using only the 48kHz dataset led to a significant performance drop for bandwidth-limited inputs. Therefore, we dynamically applied a low-pass filter to both clean and noisy speech for each batch item during training.
• Model configurations for the 48 kHz version differ slightly from the 16 kHz counterparts:
  • We increased n_fft from 512 to 1024.
  • We changed H (hop_size) and F (frequency for the RNNFormer layers) (Table 4).
  • For linear layers in pre- and post-RNNFormer, instead of using fixed weights, we made them learnable.

Table 3. Training datasets at the sampling rate of 48kHz.

Dataset		#files	total length (H:M:S)
Clean Speech	arabic speech-train			1813	3:49:04
	VCTK-0.92 (except p232 and p257)	86638	81:21:45
	Korean multispeaker TTS-traina	120044	100:00:01
	Multilingual (KO, EN, ES, JP)-traina	357847	1000:00:07
	Korean emotional TTS-traina	97342	100:00:03
Noise	DEMAMD-trainb	6240	17:20:00
	DNS-Challenge-noisec	1169	3:14:02
	FMAc	19	0:09:29
	MSRBench-targetc	821	2:16:50
	Spheres-stereomixd	1274	1:46:04
	TUT-urban-2018-dev	8640	24:00:00
	WHAM-noisec e	9279	25:41:52
	URGENT 2025-simulated wind noise	200	0:50:00

^a Dataset downloaded from 'The Open AI Dataset Project (AI-Hub, South Korea)'. Exporting these datasets outside of South Korea is prohibited. We randomly sampled a subset of each dataset as our code is not optimized for handling large-scale data.
^b Following the Voicebank-Demand recipe, we used "DKITCHEN, DWASHING, NFIELD, NPARK, NRIVER, OHALLWAY, OMEETING, PCAFETER, PRESTO, PSTATION, STRAFFIC, TCAR, TMETRO" for training. Other subsets are included in the Voicebank-Demand-test, which was used for our model evaluation. Also, we segmented each audio file into 10-second clips.
^c We filtered out audio files which didn't contain active segments in the 22.05~24kHz band.
^d We mixed all instruments into a single track. Then, we segmented it into 5-second clips.
^e We segmented each audio into 10-second clips.

Table 4. Model configuration comparison.

size	Para. (K)		MACs		H		F		RTF (M5)a
	16khz	48khz	16khz	48khz	16khz	48khz	16khz	48khz	16khz	48khz
Tiny	22	28	60M	177M	256	512	16	24	0.0058	0.0122
Base	91	101	262M	750M	256	512	24	36	0.0110	0.0261
Small	194	207	664M	1822M	256	512	36	48	0.0189	0.0421
Medium	492	512	2.9G	8.0G	160	320	48	64	0.0386	0.1347
Large	1105	1132	12.0G	32.4G	100	200	64	96	0.1052	0.4239

^a RTF measured on a single thread of a laptop CPU (Apple M5, MacBook Air)