Cortical Echoes
Amy Karle & Eduardo Reck Miranda
2026
Cortical Echoes: An Interface for Living Computation is a live performance-installation by Amy Karle and Eduardo Reck Miranda that transforms living neural activity into music, synthetic speech, and responsive visual environments. Bridging bioart, biological computing, AI, neuroscience, and experimental media, the project makes emerging forms of living intelligence publicly experiential, ethically legible, and culturally resonant.
Working with living neurons cultured on multi-electrode arrays, the project creates a three-way dialogue between human performers, digital translation systems, and living neural activity, transforming biological signals into evolving music, synthetic speech, and responsive visual environments.
The work uses musical input mapped to stimulation patterns inspired by the tonotopic organization of the auditory cortex. Neuronal responses are captured as spike activity and translated into audiovisual form, including sound structures, speech synthesis, raster-based visualizations, and immersive light environments. What emerges is not a representation of intelligence, but an encounter with living computation as process, variability, relation, and response.
Cortical Echoes explores how creativity, agency, and authorship shift when the responding system is alive. It asks how biological intelligence, machine systems, and human intention might co-create, and what kinds of ethical, aesthetic, and cultural frameworks are needed as post-silicon forms of intelligence begin to enter public life. The project positions art as a civic and sensory interface for emerging biocomputing, making frontier research experiential, visible, and open to reflection.
Musical Output from Neuronal Culture
Synthetic Speech Output from Neuronal Culture
Selected visual outputs from Cortical Echoes: An Interface for Living Computation, an art-lab performance that operates as a biological computing system using living neural networks cultured on multi-electrode arrays. Neural spike activity is recorded, interpreted, and translated in real time into generative music, synthetic speech, and responsive visual environments, forming a closed-loop interface between biological and computational processes. These images trace that continuum, from microscopy of neuronal tissue and electrophysiological data to emergent audiovisual forms and the performative environment, making living computation perceptible, legible, and experientially accessible.
Cortical Echoes
A closed-loop art-lab system coupling living neuronal activity, computational translation, and human performers and publics in time.
This section presents the project as a technically specific, experimentally structured, and publicly legible inquiry into living computation. It distinguishes measured activity from derived analysis, rendered outputs, and claims under testing; it shows explicit rule-based mappings from spike activity into music and synthetic speech; and it builds synchronized observation, recording, comparison, and public encounter directly into the system architecture.
Closed-Loop Architecture and Neural Array Observatory
The living substrate is the measured core. Performer and public signals enter a translation layer, stimulate the multi-electrode array, generate observable recruitment and spike activity, and return as rendered audiovisual form and new conditions for interaction.
Performer Input
Human-generated signals initiating the closed loop.
- Voice
- Instrument
- MIDI / gesture
- Timed stimuli
Public / Environmental Input
Additional signals placing publics inside the inquiry.
- Sound
- Speech
- Ambient conditions
- Optional gentle physiological input
Translation Layer
Computational mediation between input, stimulation, analysis, and rendering.
- Feature extraction
- Stimulation mapping
- Spike capture
- Timing / event analysis
- Routing to sound, speech, and visuals
Rendered Outputs
Public-facing audiovisual forms generated from spike-conditioned mappings.
- Music
- Synthetic speech
- Responsive visuals
- Projection / light
- Microscopy / live lab feed visualization
Feedback to Performance Environment
Outputs re-enter the room as new conditions for interaction.
- Perception
- Adjustment
- Re-entry
- Co-creation
Evidence and Comparison
The system is also an inquiry architecture, not only an expressive interface.
- Session-aligned recording
- Cross-run comparison
- Control conditions
- Publicly legible witnessing
Rule-Based Translation Grammars and Output Mappings
Inspectable mappings from spike activity into speech-like and music-like forms. These branches make explicit where biological activity is measured, where rules are applied, and where artistic rendering enters.
Spike-Conditioned Speech Rendering
Grammar symbols
A = article
N = noun
J = adjective
R = verb
Grammar set
Ten constrained sentence grammars. Under the interrogative rule, order may reverse.
Phonology
Consonant inventory, vowel inventory, word templates, stress rules, variable timing, optional vocalisation.
Interpretation
Not decoded semantic language. A transparent rule-based rendering from spike-conditioned selection and linguistic generation.
Spike-Conditioned Music Rendering
Default timing
2 seconds of raster = 1 measure of 4/4.
Default channels
Channels 20–25 for the current music branch.
Current output
Fixed pitch in the current version; timing and polyphonic distribution arise from spike events.
Interpretation
Explicit rhythmic transcription from spike timing, not a claim that neurons contain musical semantics.
Neural and Translation-State Visual Outputs
A responsive visual translation layer that makes spike timing, channel recruitment, temporal structure, and synchronized system state perceptible in public space without implying unmeasured neural connectivity or decoded meaning.
Visual Inputs and Mapped Variables
Measured inputs
Spike timestamps, active channels, stimulation regions, repetition structure, and synchronized session timing.
Derived inputs
Density, accumulation, recruitment pattern, temporal contrast, comparison across runs, and translation-state markers.
Current visual sources
64-channel spike data, raster plots, stimulation-state information, musical state, speech-state information, and synchronized event logs.
Scientific boundary
The visual field represents measured or derived variables, not a literal synapse-by-synapse connectome.
Responsive Visual Translation Behaviors
Primary visual field
The 64 channels map to an 8×8 field in which each channel appears as a point of light that pulses with spike activity.
Recruitment layer
Direct stimulation regions and wider network recruitment remain visually distinct so spread is legible over time.
Additional layers
Additional layers reveal rhythmic structure, grammar-state transitions, language-code traces, and comparison between live and replayed runs.
Temporal legibility
Accumulation, decay, contrast, and recurrence make timing, repetition, and change across sessions perceptible.
Display and Installation Layers
Source view
The installation presents raster-derived and source-linked visual states as part of the same public signal chain.
Translated field
The translated visual field presents pulse activity, recruitment, density, temporal persistence, and comparison logic as one coherent environment.
Overlay logic
Language-state, rhythmic-state, and session-state overlays remain available within the same visual architecture.
Public legibility
The installation keeps cause, response, and change over time readable, so the visual system functions as evidence and encounter at once.
Public Encounter and Process Visibility
How publics stand in the questions.
The Art-Lab
The art-lab is the shared operational environment in which living neuronal activity, computational translation, performance, responsive media, and synchronized recording are made co-present. It is the condition through which inputs, outputs, comparisons, and candidate adaptation can be witnessed, tested, and interpreted in time.
Public Encounter Layer
- Contribute input through voice, sound, or other bounded participatory signals
- Witness change over time across repeated interactions
- Compare response patterns, live versus replayed, biological versus AI-only
- Interpret what counts as reactivity, adaptation, or co-creation
- Reflect on agency, care, authorship, and what kind of computing this makes thinkable
Material and Process Layer
Synchronized Observation, Recording, and Experimental Design
A lightweight, time-synchronized framework for live witnessing, later analysis, iterative development, and curatorial archiving. Experimental structure, comparison logic, and evidence remain legible within the same public-facing system.
Session Design
Baseline spontaneous activity
Reference state before intentional input.
Repeated controlled stimulus block
Matched inputs for comparison across trials.
Contrasting stimulus block
Differential response structure under clearly distinct input conditions.
Live interaction block
Performer-driven improvisation and exchange within the closed loop.
Optional public input block
Bounded participatory entry without collapsing the comparison structure.
Post-stimulus recovery block
Drift, persistence, or return toward baseline after interaction.
Measures and Comparisons
Directly measurable now
Spike count and density, active channels, spike timing distributions, channel recruitment, grammar selection frequency, quantization outcomes, output timing, and comparison-ready session markers.
Adaptation-testing framework
Repeated matched trials, defined task metrics, retention checks, cross-run response comparison, and candidate improvement over time.
Candidate adaptation markers
Repeatable changes across matched trials, improvement on defined task metrics, retention over time, cross-run drift, and related changes in density, recruitment pattern, output variability, or task performance, in live-culture conditions only.
Scientific caution
The simulator validates workflows and routing. Strong claims about plasticity, task-dependent learning, or adaptive change require live neuronal cultures, repeated matched input, and predefined performance metrics.
Synchronized Data Layers
Every meaningful event receives a shared session reference: UTC wall-clock time, monotonic time, session UUID, condition ID, source ID, and payload. Native platform recordings remain source-of-record where available; derived manifests store aligned events, parameters, and annotations.
Neural data
Spike timestamps, channel identifiers, raster windows, recruitment structure, response summaries, and related session-linked neural measurements.
Stimulation / input
MIDI events, performer actions, stimulus commands, input onset / offset, and public-facing input states where active.
System state
Simulator or live mode, session ID, condition labels, parameter sets, version state, and latency context where available.
Translation logic
Selected grammar, question-state logic, selected channels or bins, quantization outcomes, visual-state mappings, and other rule-state decisions.
Audiovisual outputs
Music-state events, speech-state outputs, responsive visual states, rendered traces, output timing, and replay-linked output references.
Human annotations and context
Performer notes, notable events, anomalies, observer annotations, environmental context, and cross-run comparison notes.
Questions the Project Asks
Ordered from the most substrate-specific and experimentally urgent to broader scientific, artistic, ethical, and cultural inquiry.
- What can biological computing uniquely do that no other computational substrate can do within practical time, energy, and data constraints?
- Can adaptive biological systems in a closed loop still be accurately described as machine learning, or does that term begin to fail when the substrate is alive?
- Is artificial intelligence the correct term when the responding substrate is living neuronal tissue rather than silicon hardware alone?
- What kind of intelligence is present here, and how can we meaningfully engage it without collapsing it into anthropomorphic myth or reductive instrumentality?
- What new forms of relation, perception, memory, or knowledge become possible when human performers, code, and living neurons are coupled in time?
- What qualifies as co-creative partnership versus mere reactivity, and what operational markers could distinguish the two?
- How can publics stand inside the inquiry as contributors, witnesses, interpreters, and ethical participants rather than passive viewers of a technical demonstration?
- What should be explored next across scientific, artistic, ethical, and cultural domains while biological computing remains open enough to question?
- How should agency, authorship, responsibility, and care shift when the system producing response is partially biological and partially computational?
- What kinds of futures become thinkable when post-silicon computation is not only efficient or novel, but culturally legible, publicly encounterable, and alive?
SELECTED RESEARCH & SCIENTIFIC CONTEXT
Cortical Echoes: An Interface for Living Computation emerges through artistic research and interdisciplinary inquiry across biological computing, neural interfaces, unconventional computation, AI music, and experimental media. The project builds on decades of research into in vitro neuronal networks, bioelectrical signaling, adaptive learning in living neural systems, and computational approaches that move beyond conventional silicon-based models. The references below offer selected context for the project’s research lineage, conceptual framework, and technological basis. Eduardo Miranda’s long-standing research in unconventional computing and sound is central to this foundation, alongside selected publications from Cortical Labs and related scientific work in biological neural networks.
-
- Miranda, E. R., Bull, L., Gueguen, F., & Uroukov, I. S. (2009). Computer Music Meets Unconventional Computing: Towards Sound Synthesis with In Vitro Neuronal Networks. Computer Music Journal, 33(1), 9–18.
- Miranda, E. R., Adamatzky, A., & Jones, J. (2011). Sounds Synthesis with Slime Mould of Physarum Polycephalum. Journal of Bionic Engineering, 8(2), 107–113.
- Miranda, E. R. (2021). Handbook of Artificial Intelligence for Music. Springer Nature Switzerland.
- Kagan, B. J. et al. (2022). In Vitro Neurons Learn and Exhibit Sentience When Embodied in a Simulated Gameworld. Neuron.
- Hogan, D. et al. (2026). CL API: Real-Time Closed-Loop Interactions with Biological Neural Networks. arXiv:2602.11632.
- Kagan, B. J. (2025). The CL1 as a Platform Technology to Leverage Biological Neural System Functions. Nature Reviews Bioengineering.
PROJECT SCOPE
Developed across biocomputing, artificial intelligence, experimental music, bioart, biodesign, neuroscience, new media art, and computational media,
Cortical Echoes is designed for performance in museums, biennials, festivals, and research-driven public platforms.Â
It brings together Eduardo Reck Miranda’s pioneering work in AI music, brain-computer music interfacing, and biological computing for sound with
Amy Karle’s practice in biofeedback, bioart, AI, visual systems, and ethical inquiry.
Together, the project proposes a new artistic and cultural interface for creativity and living intelligence.
CREDITS & SUPPORT
Conceived and developed by Amy Karle and Eduardo Reck Miranda
Visual art and design: Amy Karle
Music and speech systems: Eduardo Reck Miranda
Software development: Moein Fahmideh Vatandoost
Developed with support from the University of Plymouth, Cortical Labs, and Conceptual Art Technologies
S+T+ARTS PRIZE 2026
https://ars.electronica.art/starts-prize/en/cortical-echoes/Â
Â
