Facet 3 · The AVK Viewport

The Three Senses

Audio · Visual · Kinesthetic — the three channels of all human experience, and every communication technology ever invented
Facet 3NeuroscienceMedia HistoryTechnology

Introduction

The AVK Viewport is Prospectus Celestial's three-channel output model: Audio, Visual, and Kinesthetic. These are not three arbitrary categories chosen by an engineer — they are the three fundamental channels through which all animal life receives and processes reality. Every piece of sensory information that has ever reached a human brain has arrived through one of these three pathways. Every communication technology ever invented routes through one or more of them.

"The medium is the message." — Marshall McLuhan, 1964. But McLuhan's insight goes deeper: the medium is not just the message — it is the channel. Audio, Visual, and Kinesthetic are the only three channels that exist.

When you declare avk="audio visual kinesthetic" in a PCML viewport, you are not configuring a software feature. You are choosing to engage all three channels of human consciousness simultaneously — the same choice made by every opera composer, every film director, every game designer, every architect who has built a space people walk through.

The Three Channels

A
Audio
Sound, music, speech, environmental audio. Processed by the auditory cortex. The fastest channel — humans detect audio events as quickly as 2 milliseconds after onset.
V
Visual
Images, text, motion, color. Processed by ~30% of the cerebral cortex — the largest dedicated sensory region in the human brain. The highest-bandwidth channel.
K
Kinesthetic
Touch, movement, proprioception, gesture. Processed by the somatosensory cortex. The most intimate channel — requires physical contact or physical action.

A · The Audio Channel

Human hearing operates across a frequency range of approximately 20 Hz to 20,000 Hz. The cochlea of the inner ear performs a remarkable physical Fourier transform: it decomposes incoming sound waves into their component frequencies mechanically, before any neural processing begins. The basilar membrane — a tapered strip of tissue 35mm long — is a biological frequency analyzer, with each point responding maximally to a specific frequency. This is hardware-level signal processing, running continuously since the first auditory organism evolved roughly 500 million years ago.

PCML Audio technologies: The @sound and @music tags route through the Windows Media Audio graph. Modern digital audio works at 44,100 or 48,000 samples per second (CD quality and professional standard respectively) — each sample a 16- or 24-bit measurement of air pressure. A three-minute song at CD quality is 31,752,000 individual measurements. The @audionode DSP chain applies mathematical transformations (convolution for reverb, parametric filtering for EQ) to these numbers in real time.

Metaphor
"The @audionode reverb effect is a cathedral built in mathematics. It takes 0.06 seconds of impulse response — the sound of a real stone space — and wraps every sound in that space's acoustic fingerprint. You hear the room without being in it."

Historical technologies of the Audio channel: Bone conduction instruments (43,000 BCE) → resonating string instruments (~3,000 BCE) → pipe organs (3rd century BCE) → printing of musical notation (1501) → the phonograph (1877) → vacuum tube amplification (1906) → magnetic tape recording (1928) → digital audio (1979, Sony PCM) → MP3 compression (1993) → spatial audio / Dolby Atmos (2012).

V · The Visual Channel

Vision is the dominant human sense by neural investment: approximately 30% of the cerebral cortex processes visual information, compared to 8% for touch and 3% for hearing. The human eye contains approximately 120 million rod cells (for low-light monochromatic vision) and 6 million cone cells (for color vision in three wavelength bands: roughly 420nm/blue, 530nm/green, 560nm/red). This three-receptor color model is directly encoded in digital displays — Red, Green, Blue subpixels — making computer screens a direct technological extension of human visual biology.

PCML Visual technologies: The @image tag loads a PNG or JPEG into a Win2D CanvasBitmap — a grid of 32-bit RGBA values residing in GPU memory. The GPU renders these at up to 60 frames per second using hardware-accelerated texture sampling, alpha blending, and transform operations. A 4K display at 60fps produces approximately 497 million pixel writes per second — the equivalent of painting the Mona Lisa (approximately 8.4 million brushstrokes) about 59 times per second.

Metaphor
"The @bitmap tag is the world's fastest mosaic artist — placing 8.3 million individual colored tiles per frame, 60 frames per second, with sub-millisecond precision. The Byzantine craftsmen who built Hagia Sophia's mosaics worked for years to place what a GPU places in a single frame."

Historical technologies of the Visual channel: Cave painting (40,000 BCE) → ceramic pictographs (~7,000 BCE) → papyrus illustration (~3,000 BCE) → mosaic (4th century BCE) → oil painting (15th century) → photography (1839) → motion pictures (1895) → television (1927) → color television (1953) → digital displays (1970s) → OLED (1987) → 4K OLED (2012) → MicroLED (2020s).

K · The Kinesthetic Channel

The kinesthetic channel is the most underappreciated — and the most ancient. Before sight and hearing evolved their present sophistication, all organisms sensed their environment through mechanoreception: pressure, vibration, chemical gradients. The human somatosensory cortex maps the body's surface in the famous "homunculus" — a distorted figure with enormous hands and lips, reflecting the density of touch receptors in those regions.

PCML Kinesthetic technologies: The @bind and @gesture tags connect physical actions (pointer movement, touch swipes, gamepad buttons, keyboard presses) to scene expressions. Modern touchscreens detect touch with a resolution of approximately 1mm² through capacitive sensing — measuring the minute changes in electrical capacitance caused by a fingertip. Haptic feedback motors in game controllers (and smartphones) provide the reverse: converting digital signals back into physical sensation at frequencies from 10 Hz to 300 Hz.

Metaphor
"The @gesture tag is a handshake protocol between the human body and the scene. When a finger swipes across glass, the scene receives a kinesthetic event and responds. This is the oldest form of communication in biology — touch returning an answer to touch."

Historical technologies of the Kinesthetic channel: Direct manipulation of objects (origins of tool use, ~3.3 million years ago) → writing instruments (stylus, brush, quill) → keyboard (typewriter, 1868) → mouse (Douglas Engelbart, 1964) → touchscreen (E.A. Johnson, 1965) → multi-touch (1982) → capacitive touchscreen (iPhone, 2007) → haptic feedback (1990s game controllers) → motion capture (1970s, now in VR/AR headsets).

AVK and the Science of Learning

The AVK model has a parallel in educational psychology: the VAK learning styles model (Fleming & Mills, 1992) proposes that individuals preferentially learn through Visual, Auditory, or Kinesthetic (physical doing) channels. While the strong form of learning styles theory is contested, the underlying neuroscience is not: different people have different cortical investments in different sensory modalities, and multi-channel instruction (engaging A + V + K simultaneously) consistently produces stronger retention than single-channel instruction.

This is why Prospectus Celestial's avk="audio visual kinesthetic" declaration is its most ambitious: it commits the scene to engaging all three channels simultaneously. A scene that uses only Visual is a painting. Add Audio and it becomes a film. Add Kinesthetic and it becomes a game — or a conversation.

Every Communication Technology, Mapped to AVK

TechnologyAVKEra
Cave painting✓40,000 BCE
Oral storytelling✓Prehistory
Written text✓3,200 BCE
Musical instrument✓✓43,000 BCE
Theatre✓✓6th c. BCE
Printed book✓1440 CE
Telephone✓1876
Silent film✓1890s
Radio broadcast✓1920
Sound film✓✓1927
Television✓✓1927
Video game (early)✓✓✓1972
Touchscreen device✓✓✓2007
VR headset✓✓✓2016
Prospectus Celestial scene (full AVK)✓✓✓2026

The Full-Spectrum Experience

Every medium in human history has been an attempt to engage more channels simultaneously. Oral storytelling (A) added gesture (K). Theatre added costume and set (V). Film added sound (A+V). Video games added interactivity (A+V+K). Virtual reality adds spatial presence (A+V+K + proprioception). The trajectory of all communication technology is the convergence toward full-spectrum AVK experience.

Prospectus Celestial is designed from the ground up for this convergence. It does not treat audio as an afterthought bolted onto a visual system, or kinesthetic input as a separate subsystem. The three channels are declared together in the @meta block and processed in parallel by the engine — because that is how human consciousness receives them.

The PCML Viewport Declaration

@meta {
    // Declare all three channels — engage the whole person
    @viewport width=1920 height=1080
              avk="audio visual kinesthetic"
              fps=60
              colorspace="sRGB" { }
}
// avk="visual"               — a painting, a book, a film
// avk="audio visual"        — radio with subtitles, a concert film
// avk="audio visual kinesthetic" — a game, a conversation, a life