Neuronal‐spiking‐based closed‐loop stimulation during cortical ON‐ and OFF‐states in freely moving mice

Summary The slow oscillation is a central neuronal dynamic during sleep, and is generated by alternating periods of high and low neuronal activity (ON‐ and OFF‐states). Mounting evidence causally links the slow oscillation to sleep's functions, and it has recently become possible to manipulate the slow oscillation non‐invasively and phase‐specifically. These developments represent promising clinical avenues, but they also highlight the importance of improving our understanding of how ON/OFF‐states affect incoming stimuli and what role they play in neuronal plasticity. Most studies using closed‐loop stimulation rely on the electroencephalogram and local field potential signals, which reflect neuronal ON‐ and OFF‐states only indirectly. Here we develop an online detection algorithm based on spiking activity recorded from laminar arrays in mouse motor cortex. We find that online detection of ON‐ and OFF‐states reflects specific phases of spontaneous local field potential slow oscillation. Our neuronal‐spiking‐based closed‐loop procedure offers a novel opportunity for testing the functional role of slow oscillation in sleep‐related restorative processes and neural plasticity.

The central aim of this study was to develop and validate the methodology for online detection of ON and OFF periods, and to investigate the possibility of neuronal-spiking-based closed-loop stimulation during spontaneous sleep in mice. The potential applications of this method include addressing the following questions.
1. The role of sleep in synaptic plasticity. In vitro evidence and experiments in anaesthetised animals suggest that pairing synaptic inputs with population ON and OFF periods leads to plastic changes in neural responses to stimulation (Bartram et al., 2017;Gonzalez-Rueda et al., 2018). This observation is important, as it suggests that a careful choice of the phase of stimulation could make sleep more restorative but, alternatively, could also lead to sleep disruption and potentially to the development of maladaptive plastic changes within the thalamocortical circuitry. To this end, a better understanding of the role of ON and OFF periods in neural plasticity, as suggested by previous work, is essential.
2. Effects of ON/OFF-states during spontaneous sleep on sensory responsiveness and processing of incoming stimuli (Massimini et al., 2005;Nir et al., 2015Nir et al., , 2017. We argue that this is critical, for example, to develop the most efficient and least disruptive stimulation protocols, and to establish whether the properties of induced slow waves differ depending on background activity. 3. Correspondence between neuronal activity and local field potential (LFP) waveforms. Finally, given that individual EEG slow waves vary greatly with respect to their origin, shape, amplitude and spatiotemporal dynamics (Bukhtiyarova et al., 2019;Massimini et al., 2004;Murphy et al., 2009;Nir et al., 2011;Riedner et al., 2011), targeting those directly with conventional closed-loop paradigms likely leads to many instances when stimulation is delivered during a suboptimal or even undesirable phase of the network oscillation. Arguably, this could influence the outcome of modulation. Therefore, obtaining a better understanding of the correspondence between neuronal activity and EEG/LFP waveforms across cortical layers will provide important refinement, both conceptual and methodological, for the approach used to target sleep slow waves.

| METHODS
All experiments were carried out in accordance with the UK Animals (Scientific Procedures) Act of 1986. All animals used in this study were C57BL/6JOlaHsd purchased from Harlan Laboratories and kept on a regular (non-inversed) 12 hr light/dark cycle. Seven male adult C57BL/6 mice (age at baseline recording 125 ± 8 days, body weight: 29.5 ± 0.8 g) were used for all experiments.

| Implants and surgical procedure
All implants were prepared manually before the surgery. For the frontal and occipital EEG recordings, silver wires were wrapped around blunted skull screws and soldered to a 90-degree connector (Pinnacle Technology, Lawrence). For the electromyogram (EMG), the end of a silver wire was bent into a U-shape and then twisted, to avoid sharp edges. This was done on two separate wires that were soldered to the above-described EEG head stage. The laminar probe (NeuroNexus Technologies; A1 Â 16-3 mm-100-703-Z16) has a ground and reference wires, each soldered to male connector pins, which could then be connected during surgery to female connector pins on the ground and reference screw, respectively. The laminar probe was stained with the dye DiI (DiIC18[3], Invitrogen) before surgery to aid the localisation of the electrode tract (Krone et al., 2021).
To induce anaesthesia, the mouse was exposed to a prefilled chamber with 4% isoflurane in medical oxygen and, once the mouse had lost the righting reflex and approached a breathing rate of approximately 80 min À1 , the animal was then transferred to a heating pad and 2-3% isoflurane administered through a nose mask at an oxygen flow rate of $1-1.5 L min À1 . After the scalp was shaved and cleaned using iodine and ethanol, the animal was transferred to a stereotaxic frame where isoflurane was administered at a concentration of 0.6-1.2% at a flow rate of $1 L min À1 throughout the surgery. At this point, Metacam ® (meloxicam, 5 mg kg À1 ; Boehringer Ingelheim), Vetergesic ® (buprenorphine, 0.1 mg kg À1 ; Sogeval UK) and dexamethasone (0.2 mg kg À1 s.c.; Boehringer Ingelheim) were injected subcutaneously and artificial tears were applied. Once the head was fixed, a rectal probe was inserted to maintain core temperature at about 37 C. The scalp was opened, and the straightness of the skull was verified by levelling bregma and lambda, and the points 1 mm lateral to bregma. To minimise the loss of implants, the skull's surface was roughened using the scalpel and etching gel, and the coordinates for implantation were marked as shown in Figure 1(a). The holes for the reference (cerebellum), ground (cerebellum or left occipital) and the two EEG screws (frontal and occipital) were drilled first, and the screws were then immediately inserted using a screwdriver. Subsequently, the hole for the bipolar concentric stimulation electrode (Plastics One; see Section 2.3 "Experimental design" below for further information) was drilled, and the electrode was carefully and slowly inserted. All screws were then fixed with dental cement Super-Bond ® (Prestige Dental Products) before a craniotomy was made for the laminar electrode. Once the bone was removed, the dura was carefully rolled back with a syringe tip and the laminar probe was immediately Longer and stricter (i.e. fewer spikes) neuronal silent periods result in a larger peak in the LFP (i), which occur progressively more rarely (j). Note the resulting trade-off between parameters resulting in a large LFP peak and parameters resulting in frequent detection events inserted until the last of its 16 contacts was below the cortical surface ( Figure 1b). The craniotomy was immediately sealed with a silicone gel (KwikSil; World Precision Instruments). The entire head stage was then cemented and the EMG wires inserted into the neck before the skin was sutured, if necessary. Animals were given subcutaneous saline injections following the surgery. After surgery, animals were carefully monitored at least once a day for 7 days, and analgesics were administered orally or subcutaneously, if necessary.

| Electrophysiological recordings
Animals were moved to recording chambers at least 3 days before the start of any recording. At least 1 day into this habituation phase was allowed before the EEG head stage was connected to a cable bridging the animal and the pre-amplifier, and another day before the laminar probe was connected to the pre-amplifier. EEG and EMG signals were routed via an S-box (Tucker Davies Technologies [TDT], Alachua, FL, USA) to a PZ-5 pre-amplifier (TDT), where they were differentially digitised (relative to the cerebellar screw or the contralateral EMG wire, respectively) at 25 kHz. The signal was then sent to a RZ-2 signal processing system (TDT), which interfaced with the synapse recording software (TDT). The RZ-2 sampled the signal down to 700 Hz (applying an adequate low-pass filter at 45% of the final sampling frequency) and stored it at 305 Hz. Whenever possible, the signal was stored in this "raw" form in addition to versions with high pass filters more suitable for online monitoring (i.e. 0.5 and 10 Hz for EEG and EMG, respectively).
The signal from the laminar probe was routed directly to the PZ-5 and sampled at 50 kHz. To obtain continuous LFP data (and limit data size), one version of the signal was down sampled to 305 Hz identical to the EEG signal. For stimulation-evoked activity, a snippet of the LFP was stored at 3.5 kHz whenever the recording software triggered a stimulation. Specifically, the snippet started 500 ms before the stimulation and was 1.5 s long. An important consequence of this procedure is that there may be a small (< 1 ms) delay between the time when the software sends out the trigger and when the current is applied by the stimulation box.
To record spiking activity, the laminar signal (at 25 kHz) was filtered between 300 and 3000 Hz, and a manual threshold was set for each channel individually. The manual threshold was initially set at least 2 standard deviations from the mean. It was then further increased until the average spike waveform (10 s running window) no longer contained noise. Whenever the signal on a given channel crossed the threshold, the timestamp of threshold crossing and a 1.56-ms-long snippet of the signal was stored at 12 kHz. This procedure has the advantage that it strongly reduces the considerable data load of recording 16 channels for days at 25 kHz. On the other hand, it irrevocably discards data, especially given than each channel typically recorded spiking activity from more than one individual neuron. In other words, some spikes are too small to trigger the threshold, while some noisy events or spikes produced by one neuron will trigger it and thus create a 1.56-ms-long refractory period where spikes produced by other neurons will be lost. Spiking activity was always processed with WaveClus (Chaure et al., 2018). This software was chosen because it was designed explicitly to perform well on single-channel recordings as well as multi-channel recordings. In contrast, most other spike-sorting algorithms are optimised for polytrode recordings (Chung et al., 2017), where a single unit is recorded on > 1 channel. Such cases are likely rare in the present recordings, given the relatively large distance between channels (100 μm).

| Experimental design
Our experimental design included cortical electrical stimulation during both waking and sleep to investigate: (a) the immediate effects of stimulation on cortical responses; and (b) to address the effects of stimulation on synaptic plasticity (Bartram et al., 2017; To this end, every mouse was subjected to at least four basic experimental conditions on separate days ( Figure 1c). Each condition began with 80 stimulations at 0.1 Hz (the current pulses were 0.1-ms squared monophasic pulses and the chosen output voltage was normally about 6-10 V) approximately at ZT 1 (1 hr after lights on). During this "baseline waking" period, mice were kept awake by providing novel objects. Following this baseline stimulation, animals were exposed to the different experimental conditions (described below) for approximately 2.5 hr. This will henceforth be referred to as the "pairing" period, because electrical stimulation was typically paired with a specific state (even though no stimulation may occur in some cases). After this pairing period, a post-pairing wake stimulation followed in all conditions. This post-pairing wake stimulation was always identical to the pre-pairing wake stimulation on all days for the same mouse (very subtle differences in baseline stimulation parameters occur in a few mice, but all variance is between mice, never within mice). As shown in Figure 1(c), the four basic conditions were: (1) sleep-mock: stimulation was targeted alternately at ON/OFF-states but the stimulation box was turned off; (2) sleep-ON; and (3) sleep-OFF where stimulation was targeted selectively at ON-and OFF-states, respectively; (4) wake-stim: the same number of stimuli were delivered as during (2) and (3), but the animal was kept awake with novel objects. The interstimulus interval was similarly constrained as during (2) and (3), but was adjusted such that the same number of stimuli was delivered in approximately the same amount of time. The number of stimulations delivered during these pairing protocols was determined by the first experimental day in each animal. The animal was allowed to sleep for up to 2.5 hr, and the only constraint on stimulation numbers was the minimum interstimulus interval (10 s), the amount of NREM sleep and the number of ON/OFF detections. Thus, the interstimulus interval was sufficiently long to prevent induction of plasticity or over-stimulation, but also sufficiently short to obtain a sufficient number of stimulations for subsequent analysis. The total number of stimulations during the pairing period varied slightly between animals but never within animals (i.e. it never varied between conditions). In all subsequent experimental days, the same number of stimulations was delivered (except for mock stimulation days). Therefore, the total duration of the experiment was kept constant at approximately 2.5 hr, but varied slightly (within $20 min) between conditions. To avoid a systematic effect of repeated stimulation, the order of the conditions was randomised, except that wake-stim (4)

| Histology
After the experiments were completed, animals were deeply anaesthetised with an intraperitoneal injection of pentobarbital (Euthanal). Once the animal reached deep anaesthesia (as verified by loss of righting, pedal and corneal reflexes), microlesions were performed to aid laminar identification of recording sites (Krone et al., 2021). For microlesions, the laminar probe was connected to an impedance testing device (NanoZ, Plexon), which was used to pass current (10 μA for 10 s) through four equally spaced channels of the laminar probe. The bottom channel was always lesioned first, as the first lesion can damage the other channels. Animals were then transcardially perfused with phosphate-buffered saline (PBS) and 4% paraformaldehyde (PFA), and the head of the animal was then stored in 4% PFA (i.e. the implant was not removed at this point, which improved the quality of histology) and moved into acidified PBS after a few days. Brains were embedded in agarose and cut into 50-μmthick coronal sections. The sections were stained with 4 0 ,6-diamidino-2-phenylindole (DAPI) and imaged using a fluorescence microscope.
The sections containing the electrode tract were identified using the red Dil fluorescence, and were imaged at 1.6, 2.5 and 5 Â magnification. The recording locations in the rostrocaudal and mediolateral dimensions were identified using the mouse brain atlas (Paxinos & Franklin, 2001). The cortical layer of each laminar contact was identi- While layer 4 is comparatively small in the primary motor cortex, it exists and can be identified as a small increase in cell density right above layer 5 (Skoglund et al., 1997;Yamawaki et al., 2014). Layer 6 was also identified based on the higher density of cells compared with layer 5.

| Scoring of vigilance states
Data were extracted from the raw data format of the recording software, resampled to 256 Hz and bandpass-filtered using custom  Figure 1g).
When an animal displayed wake-like activity for less than 4 consecu- there was sleep-like activity in the 2 s before or after stimulation.
Vice versa, if stimulation was aimed at NREM episodes, activity was scored as REM or waking if the activity 2 s before or after the stimulation resembled the respective state. The same "over-sensitive" procedure was applied with regards to artefacts. We found that stimulation did not have a major effect on the amount of vigilance states, and > 95% of stimulations targeted sleep as intended.

| Statistics
The experimental design of this study posed several statistical challenges. Most notably, each mouse experienced several treatments, and observations were often nested (e.g. multiple channels, per mouse and several mice per condition). To address these challenges, we used linear mixed effects (LME) models (Harrison et al., 2018). This method has several advantages, most notably it can account for the abovementioned nested nature of experiments and it can readily handle missing data points (e.g. a noisy or unresponsive channel on 1 day).
Each time an LME was used, all assumptions of LMEs (independence, homogeneity of variance, normality of error, and linearity) were visually inspected using plots (e.g. QQ plots). To test for significance, we used Matlab and R-studio to fit a model with and without the relevant parameter (e.g. condition) and compared the models using the loglikelihood ratio (LLR) test. If the result was significant we ran post hoc Tukey contrast in R-studio.

| Real-time detection of ON-and OFF-states during sleep in freely moving mice
We chronically implanted seven mice with frontal and occipital screws to monitor the EEG, and with two wires in the neck muscles to measure the EMG. For neuronal activity recording, we implanted a 16-channel laminar probe into the primary motor cortex (M1; Figure 1a). To detect ON-and OFF-states online, spikes were summed across all channels of the laminar probe (Figure 1d). OFF-states were detected when the running sum of spikes was below a certain threshold (usually below 1 or 2 spikes) for a sufficient amount of time (50-125 ms). An ON-state was defined as a period of high firing (10-30 Hz) for a prolonged period of time (same duration as OFF-state), following an OFF-state. A challenge for this procedure is the trade-off between speed and accuracy and the trade-off between sensitivity and selectivity. Furthermore, the optimal parameters are not uniform across animals, in part due to different numbers of neurons recorded by each laminar probe. Therefore, we used a baseline recording of each mouse to simulate ON/OFF-state detection with differing parameters. As expected, increasing the minimum duration of OFF/ON-states leads to detection of larger amplitude slow waves in the LFP but also to fewer detections of ON/OFF-states ( Figure 1i,j), as has been previously reported (McKillop et al., 2018;Vyazovskiy, Olcese, et al., 2009). We surmise that increasing the minimal duration of OFF/ON-states leads to an increased chance to detect a state towards its very end.
As expected, we found that OFF-state detection was always preceded by a period of neuronal quiescence, whereas ON-state detections were preceded by increased spiking (Figure 2a). Upon detection of OFF and ON periods, the probability to transition out of the detected state began to increase logarithmically (Figure 2b,c). Importantly, the detection of ON and OFF periods based on neuronal spiking was on average asso-  with an average probability of 51 ± 18% (mean ± SD, n = 7 mice with 16 channels each) across layers 1, 2/3, 5, and there was a significant effect of layer on response probability (p < 0.001, LLR test [dF = 3, χ 2 :37.9]). The spiking response generally involved a period of increased firing, followed by a period where spike rates fell below the spontaneous rates. The increased firing rate began on average 3.66 ± 0.93 ms (mean ± SD; n = 72 channels from 7 mice) after stimulation, and started significantly later in layer 1 compared with L5 and L6 (Figure 3b). Notably, in every experiment there was at least one channel that significantly responded within 1 ms of stimulation (mean time to first responsive time bin in any channel across mice: 1.59 ± 0.69 ms [mean ± SD]). This could be due to unaccounted stimulation-induced noise or antidromic activation. The spiking response peaked between 5 and 10 ms and, on average, lasted until 10.7 ± 2.13 ms (mean ± SD) after the stimulus.

| Neuronal responsiveness differs between ON and OFF periods
The LFP response was closely related to the spiking response, but appeared to be slightly delayed. Averaged across all responsive channels, the LFP had a negative peak of 314 ± 172 μV at 8.7 ± 1.85 ms after the stimulation (mean ± SD; n = 7 mice with 16 channels each).
There was no evidence for an interaction between condition and time in the EEG/LFP (LLR, χ 2 (18) = 16/12, p = 0.59/0.8). Post hoc comparisons in the EEG suggested that OFF-state pairings were associated with significantly reduced SWA, compared with ON-state pairings and mock pairings (Tukey contrasts, p < 10 À5 for both comparisons in the EEG). This suggests that, independent of when the measurement was taken, the OFF-state pairing condition always displays lower SWA.
This contrasts slightly with the visual impression that the first and last time bins are not different, and is likely due to insufficient power.
The slow oscillation is not the only network phenomenon during natural NREM sleep, and other events, such as spindles, have been associated with plasticity. To test the effect of stimulation on other frequencies we calculated the difference between the average power spectra (again including only 4-s epochs without stimulation) across conditions (Figure 4c,d). As expected from the previous results, there was a significant interaction between condition and frequency in the LFP (χ 2 (1) = 36.9, p = 10 À9 , LLR test) and in the frontal EEG  (Figure 4d), there was also evidence that ON and OFF pairing had differential effects on power spectra (interaction between condition and frequency: χ 2 (79) = 685.8, p = 10 À16 , LLR test, LME models with channels and mouse as nested random effects).

| Using closed-loop ON/OFF stimulation to estimate effect sizes of sleep-dependent plasticity
One important application of the approach we describe here is to address the hypothesis that pairing an input to cortex with ON-and OFF-states has differential effects on synaptic strength. To this end, we recorded LFP and neuronal responses to contralateral electrical stimulation in awake mice exploring objects, and used the magnitude of this response in the LFP and MUA as a proxy for synaptic strength (Fisher et al., 2016;Vyazovskiy, Olcese, et al., 2009). We delivered 80 stimulations (0.1 Hz) before and after each of the following different pairing protocols shown in Figure 1(c): stimulation during ON-states, stimulation during OFF-states, mock stimulation (stimulation turned off), or during waking (novel objects were given to promote wakefulness when necessary).
The effect of the four different pairing conditions (referred to as "condition") on the change in LFP peak amplitude from pre-to post-pairing wakefulness ( Figure 5) was assessed with LME models of the form: The model supported a significant effect of condition on the change in LFP peak amplitude (Figure 5a). This was true for both the relative change (e.g. V post /V pre ) and the absolute change (e.g. V post À V pre ; Figure 5ab). However, post hoc tests only yielded significant differences for the relative change, suggesting ON pairings are associated with a stronger decrease in amplitude compared with all other pairings save OFF-state pairings (Tukey-adjusted contrasts for difference in β-values ± SE: ON-mock: À7.2 ± 2.4%, p = 0.03; ON-wake: À10.39 ± 2.4%, p < 0.001; ON-OFF: À4.71 ± 2.4%, p = 0.37; Figure 5a). There was no evidence for a significant interaction between baseline amplitude and condition. While wake pairings had a trend towards increasing the response, this was not significant. We In summary, our data suggest that different neuronal states have varying effects on neuronal plasticity. However, using the methodology in the present paper, the differences were subtle (about 5%), and would thus require a substantially bigger sample size to allow more robust conclusions.

| DISCUSSION
Here we developed a method for online detection of cortical ON/OFFstates during spontaneous sleep in freely-behaving laboratory mice.
While closed-loop stimulation during slow waves is becoming increasingly popular, studies are typically based on the LFP or EEG signals only (Bellesi et al., 2014;Fattinger et al., 2019;Ngo et al., 2013;Schneider et al., 2020), which only indirectly reflect underlying network ON-and OFF-states (Thomas et al., 2020;Timofeev, 2013). One biologically effective means for closed-loop stimulation relies on setting an (adaptive) negative threshold to detect presumed OFF-states, and then targeting stimuli into the subsequent UP state by using the average delay between positive and negative peaks for each individual (Ngo et al., 2013). This method affects memory (Ngo et al., 2013), changes SWA, and influences the immune system (Besedovsky et al., 2017).
However, no studies until now have undertaken a direct online targeting of neuronal network ON-and OFF-states during sleep.
Our key conclusion is that online targeting of ON and OFF periods based on spiking activity results in a reliable detection of specific phases of LFP slow waves. Our study is consistent with the existing knowledge that spontaneous LFP and EEG slow waves, on average, correspond to a general reduction or a complete cessation of neural spiking, reflecting population OFF periods. It should be noted, however, that sleep has properties of a local process (Krueger et al., 2008) and, arguably, the neocortex is never entirely in an ON or OFF period Siclari & Tononi, 2017;Timofeev, 2013).
Therefore, targeting a specific phase of a slow wave in one cortical region will be likely associated with targeting a differentand thus potentially suboptimalphase in a different area of the brain. The consequences of such differential manipulations of slow waves in different cortical areas remain to be determined. We further observed that the evoked responses in the LFP began and peaked in deeper layers before the superficial layers. In the MUA, this trend was much less clear and likely present in only a subset of animals. An early response in deeper layers would be consistent with a strong innervation of layer 5 by callosal projections (Petreanu et al., 2007). However, such a pattern is also conceivable via polysynaptic pathways It is not yet clear whether changes in firing rates between the awake and sleep conditions are causal for synaptic changes, but our previous work indicates that pairing synaptic inputs with ON-states would weaken these inputs more strongly than pairing them with OFF-states or waking activity (Bartram et al., 2017). To test this hypothesis, here we used an experimental paradigm modified from   showed a clear trend to increase the evoked response compared with other conditions. Indeed, the time of peak was significantly delayed by wake pairings, which could support the notion that wake pairing has a significant effect of its own. In the MUA, ONstate pairings and wake pairings seemed to increase baseline firing rates. After correcting for changes in baseline firing, there was a significant effect of condition on the change in the mean number of evoked spikes. However, there was no evidence that ON-state pairings led to a weakening. Overall, plasticity in the MUA appeared to be very subtle and did not show any conclusive directionality. We surmise that due to the well-known variability arousing (Segundo et al., 1955) or sleep-promoting (Akert et al., 1952) effect, or alter sleep intensity (Landsness et al., 2011). To assess this, power spectral density was assessed in all 4-s epochs that did not include a stimulation. This analysis revealed that OFF-state pairings significantly decreased SWA activity compared with mock and ONstate pairings, in a manner not linearly dependent on time in both the LFP and EEG. Furthermore, frequencies above approximately 11 Hz had increased power in the OFF-compared with ON-state pairings.
This strongly indicates a direct effect of OFF-state pairings on sleep oscillations. The shift from lower to higher frequencies seems more consistent with arousal than with local changes in SWA. Our findings do not fully agree with previous studies using closed-loop stimulation.
For example, Ngo et al. (2013) calculated spectra across all 4-s epochs during the pairing period, and found an increase in EEG SWA when auditory stimulation was targeted to the UP state and a decrease when the DOWN state was targeted. However, when epochs including stimulation were excluded from the latter analysis, the effect was no longer evident.
In summary, our study provides important new data demonstrating feasibility of in vivo targeting of neuronal OFF and ON periods in micethe network counterparts of EEG or LFP slow waves. This method does not only represent a proof-of-concept that will inform translational studies, but it also establishes a new model for investigating the functional role of the slow oscillation in offline sensory processing and synaptic plasticity.