# Stop Typing, Start Talking: Voice Dictation in Jodit's AI Assistant Pro

Your hands are busy holding coffee. Your brain is full of brilliant prose. And the keyboard? The keyboard is a 60-words-per-minute bottleneck between the two. So in Jodit **4.12.33** we taught the AI Assistant Pro plugin to listen: there's now a microphone button in the prompt box, you talk, and the words show up.

The best part: if you're on **Jodit Cloud**, you don't wire up *anything*. No speech-to-text account, no backend proxy, no API key juggling. The same cloud key that powers your AI assistant powers your microphone too. Let's start there.

## Prerequisites

- Jodit Pro **4.12.33+** with the `ai-assistant-pro` plugin
- A **Jodit Cloud** API key (the fast path), or your own transcription backend (the DIY path)

## The easy path: Jodit Cloud (zero plumbing)

The AI Assistant Pro plugin ships with a built-in provider for **Jodit Cloud** at `cloud.xdsoft.net`. You hand it the cloud endpoint and your key once, and it configures *everything* — text generation, autocomplete, **and voice dictation** — for you:

```typescript
const providerOptions =
    Jodit.defaultOptions.aiAssistantPro.getAIProviderOptions(
        'jodit-ai-adapter',
        {
            url: 'https://cloud.xdsoft.net',
            apiKey: 'YOUR-CLOUD-KEY'
        }
    );

Jodit.make('#editor', {
    aiAssistantPro: {
        ...providerOptions
        // voice dictation is already ON.
    }
});
```

That's the whole setup. The mic button appears, the browser asks for the microphone, and you start dictating. Behind that one spread, the provider quietly set three options for you:

```typescript
// What getAIProviderOptions('jodit-ai-adapter', …) injects:
{
    voiceInputEnabled: true,
    voiceInputUrl: 'wss://cloud.xdsoft.net/v1/ai/transcribe',
    voiceInputApiKey: 'YOUR-CLOUD-KEY'
}
```

Notice it even rewrote `https://` → `wss://` and appended the transcription route. One key, one URL, derived automatically.

### Why route voice through Jodit Cloud?

- **Your STT key never touches the browser.** Audio streams to Jodit Cloud over a WebSocket; the cloud holds the provider credentials and forwards to the speech model. DevTools snoopers get nothing.
- **No second account.** You don't sign up for a transcription provider or manage its billing. Your Jodit Cloud key already covers it.
- **Usage is metered as credits.** Voice consumes from a dedicated `ai-audio` budget, with per-minute / hour / day / month limits, right next to your text and token usage in the dashboard.
- **One integration, every capability.** Text, autocomplete, and voice all ride the same provider config. Flip the cloud on, get all three.

If you want to nudge the defaults, the per-editor `voiceInput*` options still win — set them *after* the spread:

```typescript
aiAssistantPro: {
    ...providerOptions,
    voiceInputLanguage: 'en',          // BCP-47 hint
    voiceInputAutoSendSilenceMs: 2000  // hands-free (see below)
}
```

## Hands-free mode (the lazy-genius button)

Set `voiceInputAutoSendSilenceMs` and the assistant becomes a walkie-talkie: speak, pause, and after N milliseconds of silence the prompt sends itself. No reaching for Enter.

```javascript
aiAssistantPro: {
    ...providerOptions,
    voiceInputAutoSendSilenceMs: 2000 // 2s of quiet → fire the prompt
}
```

Leave it `null` (the default) and you send manually like a responsible adult.

## What actually happens when you talk

Cloud or not, the client side of the pipeline is the same, and it's worth a peek. The plugin captures your microphone as **mono PCM16 @ 24 kHz** — the lingua franca of streaming-transcription APIs — and streams those frames over a WebSocket to the transcription endpoint. Partial results come back as you speak (`onInterim`), finished phrases are committed at your cursor (`onFinal`).

The audio capture is a reusable helper you can import directly:

```typescript
import { MicrophoneStreamer } from 'jodit-pro/plugins/ai-assistant-pro/voice';

const streamer = new MicrophoneStreamer((frame) => {
    // frame: an ArrayBufferView over PCM16 little-endian samples
    socket.send(frame); // ship it upstream
});

await streamer.start(); // prompts for mic; rejects if denied
// ...later...
streamer.stop();        // tears down the AudioContext + tracks
```

Under the hood it's the classic Web Audio dance — `getUserMedia`, an `AudioContext`, and a node that resamples to 24 kHz and converts the browser's `Float32Array` to 16-bit PCM with the usual clamp-and-scale:

```typescript
function floatTo16BitPCM(input: Float32Array): Int16Array {
    const out = new Int16Array(input.length);
    for (let i = 0; i < input.length; i++) {
        const s = Math.max(-1, Math.min(1, input[i]));
        out[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    return out;
}
```

> ⚠️ **A bundler gotcha we learned the hard way:** functions that get serialized or shipped to odd contexts hate modern syntax — `for...of`, object spread, and array destructuring get rewritten by SWC into module-scoped helpers that vanish at runtime. Classic `for` loops and explicit indexing are boringly reliable. Boring is a feature.

## All the knobs (flat, no nesting)

Every voice setting is a top-level `aiAssistantPro` option — no nested config object to memorize:

```typescript
interface VoiceOptions {
    voiceInputEnabled: boolean;          // master switch (default: false)
    voiceInputUrl?: string;              // transcription WebSocket endpoint
    voiceInputApiKey?: string;           // key for that endpoint
    voiceInputDefaultModel?: string;     // e.g. 'gpt-4o-mini-transcribe'
    voiceInputLanguage?: string;         // BCP-47 hint, e.g. 'en'
    voiceInputSilenceTimeoutMs?: number; // server VAD silence window
    voiceInputAutoSendSilenceMs?: number | null; // hands-free send
    voiceInputApi?: VoiceEngineFactory;  // bring your own engine (below)
}
```

With Jodit Cloud the first three are filled in for you. The rest are yours to tune.

## The DIY path: bring your own engine

Maybe you self-host, or you already have a transcription provider you love. The plugin does **not** care who transcribes your audio — it owns the *experience* (mic button, recording pulse, cursor-aware insertion, auto-send) and delegates *transport* to a tiny interface, `voiceInputApi`:

```typescript
// The whole contract.
interface IVoiceEngineCallbacks {
    onReady?(): void;               // audio is live
    onInterim?(text: string): void; // partial transcript
    onFinal?(text: string): void;   // committed phrase
    onError?(message: string): void;
    onEnd?(): void;
}

interface IVoiceEngine {
    start(): void;
    stop(): void;
}

type VoiceEngineFactory = (
    callbacks: IVoiceEngineCallbacks
) => IVoiceEngine;
```

The button calls your factory, hands it five callbacks, and gets back something it can `start()` and `stop()`. Here's a full engine that talks **directly** to OpenAI's Realtime API from the browser — handy for local demos — reusing the same `MicrophoneStreamer`:

```typescript
import { MicrophoneStreamer } from 'jodit-pro/plugins/ai-assistant-pro/voice';
import type { VoiceEngineFactory } from 'jodit-pro/plugins/ai-assistant-pro/voice';

export function openAIRealtimeVoiceEngine(
    getApiKey: () => string,
    model = 'gpt-4o-mini-transcribe'
): VoiceEngineFactory {
    const toBase64 = (frame: ArrayBufferView): string => {
        const bytes = new Uint8Array(
            frame.buffer, frame.byteOffset, frame.byteLength
        );
        let binary = '';
        for (let i = 0; i < bytes.length; i++) {
            binary += String.fromCharCode(bytes[i]);
        }
        return btoa(binary);
    };

    return (callbacks) => {
        let ws: WebSocket | null = null;
        let streamer: MicrophoneStreamer | null = null;
        let segment = '';

        return {
            start() {
                const key = getApiKey().replace(/\s+/g, '');
                ws = new WebSocket(
                    'wss://api.openai.com/v1/realtime?intent=transcription',
                    ['realtime', `openai-insecure-api-key.${key}`]
                );

                ws.addEventListener('open', () => {
                    ws!.send(JSON.stringify({
                        type: 'session.update',
                        session: {
                            type: 'transcription',
                            audio: { input: {
                                format: { type: 'audio/pcm', rate: 24000 },
                                transcription: { model },
                                turn_detection: {
                                    type: 'server_vad',
                                    silence_duration_ms: 200
                                }
                            } }
                        }
                    }));
                    callbacks.onReady?.();

                    streamer = new MicrophoneStreamer((frame) => {
                        if (ws?.readyState === WebSocket.OPEN) {
                            ws.send(JSON.stringify({
                                type: 'input_audio_buffer.append',
                                audio: toBase64(frame)
                            }));
                        }
                    });
                    streamer.start().catch(
                        () => callbacks.onError?.('Microphone denied')
                    );
                });

                ws.addEventListener('message', (event) => {
                    if (typeof event.data !== 'string') return;
                    const data = JSON.parse(event.data);
                    switch (data.type) {
                        case 'conversation.item.input_audio_transcription.delta':
                            segment += data.delta || '';
                            callbacks.onInterim?.(segment);
                            break;
                        case 'conversation.item.input_audio_transcription.completed':
                            callbacks.onFinal?.(data.transcript || segment);
                            segment = '';
                            break;
                        case 'error':
                            callbacks.onError?.(data.error?.message || 'error');
                            break;
                    }
                });

                ws.addEventListener('close', () => callbacks.onEnd?.());
            },

            stop() {
                streamer?.stop();
                streamer = null;
                if (ws && (ws.readyState === WebSocket.OPEN ||
                           ws.readyState === WebSocket.CONNECTING)) {
                    ws.close();
                }
            }
        };
    };
}
```

Wire it in with `voiceInputApi`:

```typescript
aiAssistantPro: {
    voiceInputEnabled: true,
    voiceInputApi: openAIRealtimeVoiceEngine(
        () => new URLSearchParams(location.search).get('TOKEN') || ''
    ),
    voiceInputAutoSendSilenceMs: 2000
}
```

> **This is a demo, not production.** The browser can't set WebSocket headers, so the key rides in OpenAI's *insecure* subprotocol — visible to anyone with DevTools. That's exactly the problem **Jodit Cloud** solves: the key lives server-side, audio is proxied, and usage is metered. If you don't want to build and babysit your own proxy, the cloud path above is the grown-up answer.

## The little touches

A few details that separate "it transcribes" from "it feels good":

- **Cursor-aware insertion.** Move the caret mid-dictation and the next phrase lands where the caret is, with smart leading whitespace.
- **No phantom re-inserts.** Dictate, send, dictate again: the second phrase doesn't drag the first one back. Each phrase recomputes its baseline from the *live* input value.
- **Autosize.** The textarea grows with your monologue up to a cap, then scrolls.
- **A blinking mic.** While recording, the button pulses red — your "I'm listening" light, no spinner clutter.

## Wrapping up

Voice dictation in Jodit AI Assistant Pro has two front doors. The wide-open one is **Jodit Cloud**: point the provider at `https://cloud.xdsoft.net` with your `apiKey`, spread the result into `aiAssistantPro`, and the mic just works — server-side keys, credit metering, no STT account, nothing to host. The side door, for the self-hosting crowd, is a swappable `VoiceEngineFactory` and the reusable `MicrophoneStreamer`.

Next steps:

- **[Get a Jodit Cloud key](https://xdsoft.net/jodit/pro/)** and turn on text + autocomplete + voice in one line
- Read the [Jodit Cloud getting-started guide](https://xdsoft.net/jodit/pro/docs/cloud/cloud.md)
- Tune `voiceInputLanguage` and `voiceInputAutoSendSilenceMs` to taste
- Watch your `ai-audio` usage tick up in the cloud dashboard

Ready to give your editor a voice today? **[Start with Jodit Cloud →](https://xdsoft.net/jodit/pro/)** — one key turns on text, autocomplete, and dictation.

Now stop reading and go talk to your editor. It's finally listening.

_Full page: https://xdsoft.net/blog/voice-dictation-jodit-ai-assistant-pro_