Curr Pediatr Res 6; (&): 3-7 ISSN 97-93 www.currentpediatrics.com A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. Zu Soh, Ryo Migita, Kayoko Takahashi 3, Koji Shimatani 4, Hideaki Hayashi, Yuichi Kurita, Toshio Tsuji Institution of Engineering, Hiroshima University, Japan. Graduate schoo of Engineering, Hiroshima University, Japan 3 Orange Studio, Japan 4 Department of Physica Therapy, Prefectura University of Hiroshima, Japan. Abstract Background: Athough music therapy has ong been recognized as an effective treatment for chidren with deveopmenta disorders, evauation of their motor behavior during therapy sessions has aways depended on subjective and quaitative methods. Additionay, music therapists may face difficuties in conveying opinions based on observations conducted in therapy sessions to parents due to a wide disparity in the characteristics of chidren s behavior in different circumstances. Objective: This piot study was conducted to examine a computer-aided evauation method for music therapy invoving the use of video cameras and severa sensors. The system is used to evauate gross motor function and response to instructions from a therapist. Methods: The experiments performed incuded a hand be-paying task and severa nonmusica tasks, such as preparation of the hand bes and returning the bes to the storage box. The evauation indices were () the strength of wrist-jerk movements, () the time of response to instructions for musica performance from the therapist, and (3) the time taken to perform non-musica tasks. Work was performed to carify the correation between the resuts of evauation with the proposed method and those of an inquiry-based approach caed the Achenbach System of Empiricay Based Assessment (ASEBA), which is a standard screening method for deveopmenta disorders. Resuts: The resuts from the proposed system were more consistent with ASEBA resuts coected from therapists than with those coected from parents. Concusion: This indicates that the method can be used as a too for conveying therapists opinions to parents using the evauated indices as objective evidence. Keywords: Computer aided assessment, Deveopmenta disorder, Music therapy. Accepted Juy 8, 6 Introduction According to a survey conducted by Centers for Disease Contro and Prevention, which is a nationa pubic heath institute of the United States, the tota number of chidren with deveopmenta disabiities is increasing on a goba basis []. To improve and support the deveopment of these chidren's abiities, eary diagnosis of disabiities and an effective means of eary intervention are desirabe []. Eary-diagnosis methods have been extensivey studied in recent years. One exampe is the Achenbach System of Empiricay Based Assessment (ASEBA), which is a too for deveopmenta disorder assessment that can be used 3 both for aduts and for chidren [3]. ASEBA is intended to provide comprehensive evauation of psychosocia adaptation and maadaptive functioning based on questions regarding the subject's behavior. In ASEBA, parents and teachers assess the chid's behavior by answering these questions, and the answers are converted into mutipe assessment scaes (such as an introversion scae and a socia abiity scae) in order to identify the characteristics and probematic behaviors of the chid. Based on the answers, then, a doctor diagnoses whether the chid with a deveopmenta disorder. The questions refer to the chid's behavior over the preceding six months, which aows monitoring of deveopment over time. ASEBA is today a standard diagnostic method for deveopmenta Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. disorders because the scores are standardized for different popuations and different cutures [3]. If a chid is diagnosed with a deveopmenta disorder, eary intervention is desirabe for the prevention of secondary disabiities and for support in forming interpersona reationships [4,5]. Severa eary-intervention methods are generay empoyed, such as cognitive behaviora therapy (which improves sef-awareness through interactive interviews), physica therapy (which supports deveopment of motor function through posture and gait training) and music therapy (which trains chidren in both sociaity and motor function through the paying of musica instruments and singing). This paper focuses on music therapy in consideration of three points that are important for chidren with deveopmenta disorders [6,7]: () Since music therapy enabes communication with chidren through music without verba anguage, intervention can be started at a very eary age. () It has been reported that, based on the eicitation of rhythmic motiity, music therapy can reduce tension and anxiety and faciitate physica exercise as we as sefexpression. (3) Music therapy group sessions are effective for cutivating consideration and the abiity to get aong with others as capacities from which chidren derive sef-esteem and a strong sense of identity. However, to the best of the authors knowedge, no objective and quantitative method for evauating the behavior of chidren with deveopmenta disorders during the music therapy sessions has ever been proposed, and evauation targets have therefore depended on the experience and subjective opinion of the therapist. More importanty, chidren s behavior generay varies widey in different circumstances, causing discrepancies in evauation between parents and therapists. As a resut, therapists face difficuties and the risk of miscommunication when deivering an opinion on a chid to the parents without supportive or objective evidence. As a first step toward soving this probem, the authors conducted a piot study on a computer-aided motor behavior evauation method for specific activities in music therapy sessions. Examination of motor behavior is a favorabe starting point for future deveopment of an evauation system for music therapy because today s rapid progress in the fied of image anaysis is expected to ead to the capacity for whoy automatic evauation in the near future. Two chidren diagnosed with Autism Spectrum Disorder (ASD) and one with Typica Deveopment (TD) participated to the study. A music therapist instructed the chidren to hit hand bes, and indices incuding hand jerk and the ag time between the therapist's instruction and the ringing of each be were evauated using simpe image anaysis and auditory anaysis methods. Athough these indices may not cover the whoe scope of music therapy, they can be considered to refect part of chidren s overa behavior in such sessions. Accordingy, evauation was performed to determine the correation between these indices and the opinions of therapists and parents based on comparison with ASEBA scores. ASEBA is advantageous because it aows evauation by parents and teachers aike based on the same protoco. This enabes carification of discrepancies in evauation resuts among evauators, which can be caused by differences in circumstances among chidren. The system aso supports testing to determine the feasibiity of this computer-aided evauation method for the conveyance of therapists opinions to parents. This paper is organized as foows: Section gives an outine of music therapy, reated work, Section 3 describes the proposed method and Sections 4 and 5 discuss the experiment and its resuts. Finay, Section 6 concudes the paper. Music Therapy Starting in the ate 96s, the effects of music therapy were demonstrated and proved through various experiments, stochastic anayses and measurement technoogies [8]. Studies were activey conducted on pediatric patients, and focused on indicators such as respiration rate and the crying behavior of chidren. It was in this context that music therapy was adapted for chidren with deveopmenta disorders such as autism, spectrum disorders and attention-deficit hyperactivity disorder. Recent studies have reveaed that music therapy sessions improve joint attention, attention span and anguage deveopment [9,]. Engineered approaches to music therapy have aso been proposed. For exampe, Oshima et a. [] proposed a system that pays the music accompany with subjects capping. Kurizuka et a. proposed a mutua adaptive system in which the therapist assists the waking motion of the patient by paying music with an optima rhythm to improve the smoothness of waking movement []. However, the purpose of these systems is to restore motor function to edery or dementia-stricken patients rather than to support the improvement and deveopment of communication skis and sociaity in chidren with deveopmenta disorders. In practica music therapy for chidren with deveopmenta disorders, the therapist sets session targets in ine with individua characteristics identified from behaviors observed in a previous session. After the session, evauation is performed and is generay video-recorded to afford a deeper understanding of the chid. The targets commony defined in private music therapy faciities incude using musica instruments propery, performing cooperative actions, singing whie fiing in missing words, using eft-arm/ right-arm/both-arm approaches to musica instruments and objects, and understanding cause-and-effect reationships. The tota number of targets can exceed. To evauate how we these targets are met, the therapist is required to capture the characteristics of both fine and gross motor functions as we as to assess response to instructions during the session [3-7]. However, this evauation method is subjective and quaitative. Toward the estabishment of an objective, quantitative evauation method and the future deveopment of a music therapy evauation system, the next section outines the technique examined in the piot study. Curr Pediatr Res 6 Voume Issue & 4
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. Cameras Computer Chid Chidren behavior mode Music therapy Video Behavior images Feature Features Anaysis and measurement Sound extraction evauation Computer-Aided Music Therapy Evauation Method Figure gives an overview of the proposed method for evauating the behavior of chidren during music therapy. The approach consists of a signa-measurement process, a feature-extraction process and a behavior-evauation process, which together are used to evauate (a) motor function and (b) response to instructions. The trajectories of the wrist during the paying of hand bes are anayzed to evauate gross motor function, and the time taken for response to musica instructions as we as for task competion are aso evauated. The proposed method is intended to test the feasibiity of the computer-aided approach for quantitative evauation representing the opinions of therapists. Once the approach is verified, modern technoogies such as Kinect can be empoyed to estabish a fuy automated evauation system. The foowing sections outine each process of the proposed method. Target Task and Evauation Target The target tasks and the evauation target were configured to represent a compromise between achieving the aims of music therapy and faciitating computer-aided evauation. The main target task was to pay the hand bes foowing the instructions of the therapist. In addition, three types of non-musica tasks were aso carried out, which are Figure. Overview of the proposed system preparing the hand bes, changing hand bes, and returning the hand bes to the storage box. Athough the goas of music therapy are not as simpe as having the patient hit a be in ine with the therapist s instructions, this task aows partia evauation of the music therapy targets described in Section. By way of exampe, gross motor function can be evauated by tracking the wrist of a chid reaching out to the bes, as such function in chidren with deveopmenta disorders may differ from that in chidren with typica deveopment. In addition, indices such as the ag time between instruction and hitting a be and the number of faied attempts may be infuenced by engagement with the task, and execution time for non-musica tasks may depend on chidren s eves of compiance. A key objective of this piot study was to determine whether these evauation targets refect ASEBA scores given by therapists and parents. Signa-Measurement Process This process invoves the use of two video cameras. One of these is instaed on the ceiing and the other is fixed beside the tabe so that the behavior of the chid can be observed from both atera and overhead viewpoints. The cameras record images at f [Hz] and sound at a frequency of f [Hz]. An instruction board with circes of eight coors corresponding to the coors of the hand bes is used, and the therapist gives instructions by pointing at Musica score for instruction Camara Desk Music bes Chair (a) Arrangement of the music bes Figure. Layout of the experimenta environment (a) Arrangement of the experimenta equipment 5 Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. y x (, ) Mask(wristband) y z (, ) Input image (X, Y) Mask image (a) From the ceiing camera Mask(wristband) coor circes of the kind shown in Figure (a). A touch sensor is attached at the center of each coor circe to capture the instruction time t teach i and record it to the computer. Here i represents the scae of the hand be, where i =,, I. Feature-Extraction Process Image processing To evauate the gross motor function of the arms from the recorded video (Figure 3), HSV components of a wristband attached to the chid are extracted and the wrist s movements are tracked in each frame. First, as shown in Figure 3, each frame of the video is converted into both a brightness component image and a mask image, which are generated by extracting the area that has HSV vaues cose to those of the wristband. Noise is then reduced by performing expansion and reduction processes on the mask image, and the contour with the maximum area is extracted and identified as the area of the wristband. The equations shown beow give the center of gravity G as the wrist position of the chid in frame, where is the tota number of frames to be anayzed. Here the center of gravity of the z axia direction was determined using images from the video camera instaed on the atera wa. X Y G = xg x, y () x M x = y = M x = y = ( ) X Y G = yg x y () y M y = z = (, ) Y' z G zg y z z ( ', ) Input image = (3) (Y, Z) (b) From the side camera g( xy, ) in Equations () and () and g ( ', ) y z in Equation (3) are pixe vaues in frame number L of the ceiing camera and atera camera, respectivey. When the pixe vaue (the vaue of the pixe extracted using the, =, = ; otherwise,. The coordinates of the x-axis of the images taken from the ceiing camera are numbered as x=,,, X[pixes], and the coordinates of the y-axis taken from the atera camera are numbered as y =,,, Y' [pixes], M being the tota number of pixes in the mask area. When there is no area with HSV components cose to the wristband, the mask image from the previous frame is used. mask image) is, g( xy) g( y z) g( xy, ) = g( y, z) = Frequency anaysis Mask image Figure 3. HSV images obtained from input images To measure the time it takes for the subject to ring each be, the power spectrum of the audio signa S(t) is computed using short-time Fourier anaysis with window width ω and overap ν. The tota power Pt i () in the frequency band of fi df to f i + df [Hz] corresponding to the i-th scae of the hand be is then cacuated. The time it takes for P i (t) to exceed the threshod of θ [%] above the power of sound in the environment is defined as the response timet chid ; that is, the time it takes for the chid i to ring the i-th hand be. Behavior Evauation Process The behavior evauation process invoves cacuation of evauation indices based on features determined from the feature extraction process. Curr Pediatr Res 6 Voume Issue & 6
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. Root mean square of jerk Gross motor function during the paying of hand bes is evauated using hand trajectories, with each hand movement between two bes being considered a reaching movement. Recent research has shown significant differences in reaching movement between chidren with typica deveopment and those with deveopmenta disorders [8- ]. For exampe, Mari et a. measured the reaching-tograsp movement of chidren with autism spectrum disorder and compared the resuts to those of chidren with typica deveopment. It was found that the two types of chidren differed in trajectory panning as we as in execution process []. It has aso been reported that chidren with typica deveopment are more sensitive to bioogica motions characterized by sma jerks []. However, to the authors knowedge, no studies have empoyed reachingmotion modes to evauate chidren with deveopmenta disorders. There are three major modes for describing reaching movement: () the minimum-jerk mode, which is based on the assumption that humans naturay seect the smoothest trajectory connecting the start and end points []; () the minimum torque-change mode, which introduces joint dynamics to the minimum-jerk mode and is based on the assumption that humans seect the motion trajectory that minimizes variation in joint torque [3]; and (3) the minimum variance end-point error mode, which is based on the assumption that humans seect the trajectory that minimizes the effect of bioogica noise generated by musce and neurona activity [4]. As this study focused on the smoothness of reaching movement, the minimum-jerk mode was empoyed to evauate the motion of chidren. This mode predicts the trajectory of reaching motion by minimizing the foowing cost function when the movement duration t f is determined []: t f 3 3 3 dx dy dz C j = dt. 3 + 3 + 3 (4) dt dt dt The anaytica soution can be derived by appying the variation method to Equation (4). As a resut, the trajectory of x-axis xsim ( t ) can be expressed by the fifth-order function 5 4 3 ( ) f ( ) xsim t = x 6τ + 5τ τ (5) where, τ = t/ tf, t is time, and x f is the end point of the reaching movement. Veocity and acceeration can be assumed to be at both the starting point and the end point. The trajectories on the y- and z-axes can be derived in the same manner. To reduce the noise component generated by image processing, a simpe moving-average K-order method is empoyed. By differentiating Equation (5), veocity can be cacuated, and the waveform of veocity in the idea reaching movement takes on a be shape. Using the mean sum square error, this be-shaped veocity was compared with the veocity actuay measured in chidren. In addition, in order to evauate the smoothness of reaching movement, the effective vaue of jerk J rms is cacuated because ess jerk indicates smoother motion. The effective jerk vaue is then cacuated based on the root mean square (RMS) over time as in the foowing equation: J rms N = j( t), (6) N t= where N is the tota number of sampes coected during a tune. Number of faiures and ag time To evauate the chid's response to the therapist, cacuation was performed to determine the number of faiures and the ag time R i between the therapist's instruction and the ringing of each be. Faiure here is defined as the chid s tapping of a different be from the one the therapist indicated. Lag time R i is cacuated by subtracting time chid t i (when the be is rung) from time t chid i (when the therapist gives the instruction). Task execution time The time required to compete non-musica tasks was aso evauated. The behaviora transit caused in a chid by the therapist's instruction can be described using an infant behavior mode previousy proposed by other authors based on the Petri-nets theory [5]. Figure 4 shows a schematic diagram of the mode that describes the states of the chid and the therapist's instructions. In Figure 4, paces P, P' and I respectivey represent the state of a chid who is on task, the state of a chid who is off task, and the instruction from the therapist. T is the transition between states in each task. The current behaviora state of the chid is represented by the token of a soid back circe, and the current instruction is represented by the token of a soid gray circe. When the therapist gives an instruction (for exampe, "Put away the hand bes"), the soid gray circe moves to the corresponding pace I. This enabes the chid's token to make a transit to the instructed pace P. If the chid does not compy with the instruction, the token either does not make a transition or makes a transition to an off-task pace. In this manner, the Petri-net-based mode can visuay describe the behavior of the chid. This means that the therapist can empoy the mode in evauating on-/ off-task states by using it to track transition times between paces. However, as determining whether the chid is on or off task requires subjective evauation, it is difficut to ceary separate the behaviora states of the chid. Accordingy, the q time T = T T required for task competion was simpy cacuated, where q represents the task number, T represents the time when the therapist gives an instruction, and T represents the time when the chid competes the task. As T q increases aong with off-task time, this index serves as an indicator for the evauation of off-task behavior. Comprehensive evauation score The proposed method outputs a comprehensive evauation score through the foowing procedure. First, the indices measured from a chid subject were compared with those 7 Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. The experimenta environment and method configuration are described in this section. coected from a typica deveopment group using t-tests. The comprehensive evauation score was then defined as the average of t-vaues where significant differences were found. As t-vaues express the distance between two groups in t-distribution, the comprehensive evauation score describes the difference from the typica deveopment group. This definition was derived from Achenbach System of Empiricay Based Assessment (ASEBA) in which behavior indicators are cacuated from the t-score compared to the typica deveopment group [3]. In the same manner, ASEBA scores were aso averaged over a indices for comparabiity to the defined scores. Subjects and Therapist Three chidren (Sub. A: 9 years od/femae/td; Sub. B: 7 years od/mae/asd; Sub. C: 7 years od/mae/asd) and a music therapist participated in the experiments. Monitoring and anaysis were carried out for each chid. The monitoring was conducted in a private music therapy faciity under parent/therapist supervision in accordance with the Decaration of Hesinki to ensure that sufficient care was taken with the chidren and to prevent their exposure to excessive risks and burdens. The ethics committee of the music therapy faciity approved the monitoring and anaysis protocos. The parents provided written informed consent for their chidren s participation in the experiments. Experiments To verify the accuracy of the proposed method and indices, the behavior of chidren in a music therapy faciity was monitored. The measurement and evauation resuts for two chidren diagnosed with autism spectrum disorder (ASD) were compared with those for a chid with typica deveopment (TD). In addition, evauation resuts from the proposed method were compared with ASEBA scores coected from the parents and the therapist. Session start I T, Task I T, I I T, I T, T, P Task T, P T, T, P T, T, T T TQ, Q TQ, Q Pq Pend TQ, Q PQ TQ, Q Tq, q T End session TQ, I TQ, end PQ Tq, q IQ Tq, q+ Pq Tq, q Task Q Tq, I Tq, q T, P Iq T, 3 P T, Task q T, I T, The music therapist who participated in the experiments is certified with the Japanese Music Therapy Association and the Certification Board for Music Therapists. q T Q Chid Instruction and stimuation Figure 4. Petri-net mode for behaviora evauation of chidren x y Chid Hand be Coor score Therapist (a) Neutra (b) Indication Figure 5. Images of a chid during experiments Curr Pediatr Res 6 Voume Issue & 8
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. Experimenta Environment Figure 5 shows the hand bes and musica score used in the experiment as we as the experimenta environment. A camera is fixed on the ceiing about.4 [m] from the foor (ceiing camera). The other camera is instaed on a fixture attached to the wa and positioned approximatey.4 [m] from the desk and.8 [m] above the foor (wa camera). The behavior of each chid during the music therapy session was simutaneousy recorded from these vertica and horizonta viewpoints. As shown in Figure 5, during the session the chid and the therapist sat on chairs facing each other across a wooden tabe (width:.9 [m]; depth:.4 [m]; height:.6 [m]) on which the hand bes were aigned. The type of hand be (ZEN-ON Co., Ltd., Tokyo: music be, coor-touch type) used for the experiment is a common instrument in music therapy and can be rung by hitting a button on its top. Experimenta Protoco The repertoire used for the experiments incuded "Mary Had a Litte Lamb" (/4 time) and "My Grandfather's Cock" (4/4 time). These two tunes were chosen because the subjects had payed them before so woud not be confused during the experiments by having to pay new tunes. Another reason was that the two tunes have different difficuty eves: "Mary Had a Litte Lamb" is a simpe tune empoying ony four notes, whereas "My Grandfather's Cock" is a reativey difficut tune that has a onger paying time and empoys eight notes. The frequency of each note of the hand bes used in the experiments is shown in Tabe ("Mary Had a Litte Lamb") and Tabe ("My Grandfather's Cock"). The experiments were arranged to require the subjects to aternatey perform non-musica tasks and a musica Tabe. Musica scaes and corresponding frequencies: "Mary Had a Litte Lamb" Musica notes Freqency [Hz] C 4 46.5 D 4 74.65 E 4 38.5 G 4 567.98 Tabe. Musica scaes and corresponding frequencies: "My Grandfather's Cock" Musica notes Freqency [Hz] C 4 46.5 D 4 74.65 E 4 38.5 F 4 396.9 G 4 567.98 A 4 76 A# 4 864.65 C 5 93 task. There were a tota of five sequentia tasks arranged as foows: () prepare the hand bes, () pay "Mary Had a Litte Lamb," (3) change hand bes, (4) pay "My Grandfather's Cock," and (5) return the hand bes to the storage box. Parameter Configuration of the Proposed Method Monitoring part For the musica task, the therapist gave instructions to the subjects by pointing at coor circes corresponding to the coors of the hand bes. A touch sensor attached to the center of the circe sensed each pointing action, and the computer received the signa from the sensor and recorded the timing of the instruction. The resoution of the camera was 7 48 [pixes], and its frame rate was f = 9 [Hz]. The samping frequency of the audio signa was f=44. [khz]. Feature extraction part For the short-time Fourier transformation, the parameters of window width ω=34.5 [ms] and overap width v = 7.3 [ms] were set. The frequency margin for detecting notes was set to df= [Hz], and the detection threshod was set to θ= [%]. It shoud be noted that when the same note was continuousy payed or when the sound from the hand be was not oud enough even though the subject had hit the be, it was difficut to systematicay determine the time at which it was rung. In such cases, the time was manuay extracted using movie-editing software (Core Corporation, VideoStudio Pro X4). Behavior evauation part The data number for the moving average was set to K=5. The end of the reaching movement x f and the time taken for its competion t f were manuay extracted from the video footage recorded. The start time T and competion time T of the action task were extracted based on the instruction messages from the therapist. Movie-editing software (Core Corporation, VideoStudio Pro X4) was used to carry out these procedures. Cacuations of jerk and ag time R i were normaized based on beats per second (bps) for each tune to compensate for variation in instruction tempo. The root mean square of jerk was cacuated for each bar of each tune and faied trias in which subjects hit incorrect hand bes were excuded from the cacuation of average time ag. In addition, the numbers of faiures were manuay counted by comparing the power P i (t) of the sound signas of the i-th note with the signas from the touch sensors that indicated the therapist's instructions. Resuts Evauation of Reaching Movement Figures 6a 6c shows exampes of video images recorded whie a subject was paying "Mary Had a Litte Lamb." 9 Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. y x (a). [s] (b).5 [s] Position y [pixe] Position x [pixe] 7 48 (c). [s] (d) History of hand movement Hand position Hand position estimated by the system Figure 6. Exampe of video images and wrist movement trajectory x (t) [pixes] y (t) [pixes] C 4 F 4 E 4 F 4 G 4 F 4 G 4 A 4 A# 4 A 4 D 4 6 5 4 3 6 5 4 3 C 4 F 4 E 4 F 4 G 4 F 4 G 4 A 4 A# 4 A 4 D 4 C 4 F 4 F 4 G 4 A 4 A 4 A# 4 A 4 D 4 G 4 G 4 z (t) [pixes] 6 5 4 3.4 9.6.5 9.8 3. 3.4 Time [s] Time [s] Time [s] (a) Sub. A (b) Sub. B (c) Sub. C Figure 7. Wrist joint trajectories Curr Pediatr Res 6 Voume Issue &
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. 6 x (t) [pixe] 5 4 3. Time [s] 9.4 6.6 Time [s] 3. 5.5 Time [s].7 v (t) [pixes/s] - - Time [s] Time [s] Time [s] a (t) [pixes/s ] 5-5 j (t) 3 [pixes/s ] - 6 4 - -4-6 Time [s] Time [s] Time [s] Time [s] Time [s] Time [s] (a) Sub. A (b) Sub. B (c) Sub. C Figure 8. Wrist position measurement resuts Measured vaue Time [s] Theoretica vaue.9 Time [s].83 Time [s].97-5 -5-5 x (t) [pixe/s] - -5 x (t) [pixe/s] - -5 x (t) [pixe/s] - -5 - - - -5 (a) Sub. A -5 (b) Sub. B (c) Sub. C Figure 9. Exampe of measured wrist veocity compared to theoretica veocity cacuated using the minimum-jerk mode -5 Figure 6d shows trajectories of the wrist position extracted using the proposed method. For comparison, the position of the wristband as extracted by visua observation is aso shown in Figure 6c. These figures indicate that both trajectories are very cose, which confirms the tracking abiity. Figure 7 shows wrist positions (x, y, z) extracted from a session. It can be seen that changes in position on the y- and z-axes were sma compared to those on the x-axis; because of this, focus was paced on motion aong the x-axis for the subsequent evauation. Figure 8 shows an exampe of timeseries variations in position x(t), veocity v(t), acceeration a(t) and jerk j(t) among subjects paying the same tune. The shaded areas in Figure 8a show the interva that required each subject to sequentiay pay bes paced more than 3 [mm] apart. In this interva, position x(t) and veocity v(t) changed the most, which confirmed that movement of the hand had been successfuy extracted. Focusing on jerk j(t), the highest ampitude of Subject A was neary, [pixes/s 3 ], which was much ower than that for Subjects B and C, whose ampitudes were as great as 4, 5, [pixes/s 3 ]. Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. Tabe 3. Number of sessions for anaysis of hand-reaching movements "Mary had a Litte amb" 6 4 4 "My Grandfather's Cock" 8 Figure 9 shows exampes of the measured veocity and the veocity cacuated based on the minimum-jerk mode in the interva that required each subject to sequentiay pay bes paced more than 3 [mm] apart. Tabe 3 shows the number of sessions used for anaysis of reaching movement. Note that data were excuded from the cacuation of jerk when () the hand position coud not be extracted because the hands were hidden by the body, or () when the subject was not wearing the wristband. In the evauation of reaching motion, additiona data were excuded from Tabe 3 when such motion was interrupted for any of the foowing reasons: () the subject scratched his or her face during the reaching motion, () the subject moved his or her hand in the opposite direction from the target be, or (3) the subject was unsure about which hand shoud be used to tap the be. The resuting numbers of reaching movements used for anaysis were five for Subject A, six for Subject B and five for Subject C. Absoute discrepancies between the measured veocity and that cacuated using the mode were cacuated for each samping time, and the mean vaues and standard deviations are shown in Figure 9. This figure suggests that athough the error trend of Subject C was sighty arger than that for Subjects A and B, no significant differences among the subjects were observed. Figure shows the average RMS of jerk j rms. It can be seen that jerk for Subjects B and C had arger RMS vaues than that for Subject A. Considering the mutipicity of probems invoved, stochastic comparison was performed using the Bonferroni method. As shown in Figure a, a significant difference with a 5 [%] significance eve between Subjects A and C was detected in the simpe tune : Not significant Root mean square error 8 6 4 - Figure. Average root mean square errors between measured veocities and those cacuated using the minimumjerk mode * ** : Significant eve 5.% (p <.5) : Significant eve.% (p <.) : Not significant * 5 J [pixe/beat rms s ] J [pixe/beat rms s ] 5 5 3 5 5 5 (a) Mary Had a Litte Lamb (b) My Grandfather s Cock Figure. Average root mean square of measured jerk Tabe 4. Number of faiures "Mary had a Litte amb" "My Grandfather's Cock" Mary Had a Litte Lamb, whie Figure b indicates a significant difference with a [%] significance eve between Subjects A and C and a significant difference with a 5 [%] significance eve between Subjects B and C in the difficut tune My Grandfather's Cock. These resuts suggest that the hand motion of Subject C became more awkward with greater tune difficuty. This may have been caused by the subject s oss of concentration in the atter haf of the tunes. Evauation of Response to Instructions Tabe 4 shows the number of faiures for each subject in each session. The data suggest that Subjects B and C tended to make more mistakes than Subject A did, which is consistent with video observations. Whie Subject A waited for and then confirmed instructions from the therapist before tapping the be, Subjects B and C tended ** Trai Trai Trai 3 3 Trai Trai 3 5 Trai 3 5 * Curr Pediatr Res 6 Voume Issue &
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. Frequency 8 6 4 Frequency 8 6 4 8 (a) Sub. A, Tria (b) Sub. B, Tria Frequency 6 4 -.8 -.6 -.4 -...4.6.8...4.6 >.6 (c) Sub. C, Tria Response time [s] Figure. Histograms of time ag between therapist's instructions and chidren's responses. "Mary Had a Litte Lamb" 5 Frequency Frequency Frequency 5 5 (a) Sub. A, Tria 5 5 5 (b) Sub. B, Tria 5 5 5 -.8 -.6 -.4 -...4.6.8...4.6 >.6 (c) Sub. C, Tria Response time [s] Figure 3. Histograms of time ag between therapist's instructions and chidren's responses. "''Mz Grandfather's Cock" 3 Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. to pay the tunes to their own rhythms without reference to the instructions from the therapist. Figures and 3 show histograms of ag time. Figure suggests that Subject A had a reativey consistent ag time in responding to the instructions when paying "Mary Had a Litte Lamb." Subject B tended to tap the be before the instructions were given, whie Subject C tended to tap we after instructions were given. The variance in ag time for Subjects B and C was much greater than that for Subject A. However, Figure 3 shows that when the subjects were paying "My Grandfather's Cock," these individua characteristics disappeared for each subject and the distribution of ag time became symmetrica and unimoda. The ag time aso increased for a subjects. Figure 4 shows the average ag time R i for each subject. Figure 4a indicates a significant difference with a significance eve of. [%] between Subjects A and C as we as between B and C in the simpe tune. In contrast, Figure 4b indicates that there were no significant differences among the subjects in the difficut tune. As "My Grandfather's Cock" (which consists of 8 notes) is much onger and more compicated than "Mary had a Litte Lamb" (5 notes), a the subjects were more carefu when paying the former and this hid the individua characteristics. *** : Significant eve.% (p <.) : Not significant ***. *** R [s /beat ] i R [s /beat ] i.8.4 -.4..8.6.4. (a) Mary Had a Litte Lamb (b) My Grandfather s Cock Figure 4. Average time ag ** : Significant eve.% (p <.) : Not significant 4 8 6 4 Task competion time [s] Task competion time [s] Task competion time [s] 4 8 6 4 4 8 6 4 (a) Task : Preparation of the music bes (b) Task : Changing music bes ** (c) Task 3 : Cearing Figure 5. Average task execution time As described above, anaysis of ag time enabes quantitative evauation of differences among the subjects. In addition, the evauation resuts suggest that a simpe tune such as "Mary had a Litte Lamb," which has fewer notes to pay, is suitabe for observing individua characteristics. Evauation of Abiity to Compete Non-Musica Tasks Figure 5 shows the average task competion time for a trias. As a resut of the t-test s assumption of equa variances in average task competion time, no significant differences were found among the subjects for Task or Task. In contrast, as shown in Figure 5c, a significant difference with a significance eve of [%] was observed between Subjects A and C for Task 3. Here, considering the mutipicity of the probems invoved, mutipe Curr Pediatr Res 6 Voume Issue & 4
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. comparisons using the Bonferroni method were performed. As Task and Task posed the cear main goa of paying the tunes, a subjects were abe to compy with the instructions. In contrast, the purpose of Task 3 was vague, which made Subjects B and C unabe to concentrate on the task. The video shows that both subjects were reuctant to compete the task and ignored repeated instructions from the therapist. These resuts indicate that the proposed indices refect the characteristic behavior of subjects in both musica and non-musica tasks. Comparison with ASEBA Scores Evauation resuts from the proposed method were compared with subjective evauation resuts based on ASEBA scores coected from parents and a therapist (Kyoto Internationa Socia Wefare Center, Kyoto: Japanese version) [3]. The parents of each subject competed questionnaire CBCL6-8 and questionnaire TRF6-8. Athough the content of these questionnaires differs somewhat, both are designed to faciitate investigation of chidren s behavior. Figure 6 shows the resuts for each evauation item. The horizonta axis in Figure 6 incudes evauation items integrated from the questionnaires, and the vertica axis incudes the scores for a evauation items. Each 7 6 Borderine Cinica range Norma range Score 5 4 3 7 6 (a) Sub. A Parent Music therapist Score 5 4 3 7 6 (b) Sub. B Parent Music therapist Score 5 4 3 Anxious / Depressed Withdrawn / Depressed Somatic compaints Socia probems Thought probems Attention probems Rue-breaking behavior Aggressive behavior Internaizing probems (c) Sub. C Externaizing probems Tota probems ASEBA indices Affective probems Anxiety probems Somatic probems Attention deficit / Hyperactivity probems Oppositiona defiant probems Figure 6. ASEBA scores coected from parents and therapist Parent Music therapist Conduct probems Suggish cognitive tempo Obsessive-compusive probems Post-traumatic stress probems 5 Curr Pediatr Res 6 Voume Issue &
A motor behaviora evauation method for chidren with deveopmenta disorders during music therapy sessions: A piot study. Average of ASEBA score 65 6 59 56 53 5 score ranges from to, and those exceeding 65 are considered characteristic of a deveopmenta disorder. The soid ine in Figure 6 represents the evauation resuts coected from the parents (CBCL6-8), and the dashed ine represents those from the therapist (TRF6-8). For Subject A, the evauation resuts from the parents were roughy consistent with those of the therapist, as shown in Figure 6a. This suggests that Subject A is within the range of typica deveopment. For Subject B, the evauation resuts from the parents showed severa high scores; specificay, six items were scored high enough to be considered indicators of deveopmenta disorder, as shown in Figure 6b. The scores from the therapist were ower than those from the parents, and a items were within the range of typica deveopment. For Subject C, the parents and the therapist shared common trends in their evauations except in the case of severa items for which the therapist gave higher scores, as shown in Figure 6c. As seen here, correspondence in evauation resuts from parents and therapists cannot necessariy be expected because ASEBA evauation is subjective. Figure 7 shows the average scores of the ASEBA evauations and the comprehensive evauation scores of the proposed method. The resuting score for Subject B was points and that for Subject C was 5.6 points. Interestingy, these outcomes are consistent with the evauation resuts from the therapist as shown in Figure 7. This suggests that the proposed method can be used to convert the subjective opinions of the therapist into quantitative and objective indices, and can therefore be used to convey the ideas of the therapist to parents. Moreover, the evauation resuts can be utiized in the design of subsequent music therapy sessions. By way of exampe, as the method indicated significant jerk in the reaching movement of Subject C, subsequent music therapy sessions shoud invove activities to improve the subject's gross motor ski. Concusion Parents Music therapist Proposed method 58.9 Sub. B 5.8 56.8 57.3 Sub. C Comprehensive evauation score Figure 7. Comparison of average ASEBA scores and number of significant differences with Sub A This paper proposes a computer-aided evauation method for specific music therapy activities and describes a piot study conducted to determine whether the evauation resuts support therapist opinions. The chid's behavior was monitored during 5.6 8 6 4 the activity using video cameras, and reated characteristics were identified via image processing and frequency anaysis of audio signas. The chid's commitment to music therapy can then be quantitativey evauated using the proposed indices, which are based on consideration of some of the goas of music therapy. A chid with typica deveopment and two chidren with deveopmenta disorders (autism spectrum disorder, or ASD), a of whom attended private piano casses, participated in the experiment. Their behavior was evauated from the viewpoints of exercise abiity and response to instructions. For a proposed indices, significant differences were observed between the chid with typica deveopment and those with deveopmenta disorders. Comparison of ASEBA scores from the therapist with the resuts of evauation using the proposed method showed simiar trends between the two. In addition, the parents wecomed the evauation resuts and the video footage provided because this information supported understanding of chidren s behavior during music therapy. As the proposed method can be used to objectivey evauate chidren s behavior, it serves as a too for converting the subjective evauation of the therapist into quantitative indices and for expaining the basis of subjective evauation to parents. As the indices refect response and motor function to carify chidren s strong and weak points, they provide the therapist with reference data for subsequent therapy sessions and support recommendations for additiona treatment, such as physica therapy. Based on this piot study, the authors pan to define more indices incorporating cumuative expertise on deveopmenta disorders, such as metrics for ine of sight and sitting posture, using recenty deveoped image processing techniques and motion capture devices. Source of Funding This work was supported by JSPS KAKENHI Grant Number 5H584. References. Baio J. Prevaence of autism spectrum disorder among chidren aged 8 years - autism and deveopmenta disabiities monitoring network, Sites, United States, : Surveiance summaries: Nationa Center on Birth Defects and Deveopmenta Disabiities, Center for Disease Contro and Prevention, United States of America 4; -.. Cohen H, Amerine Dickens M, Smith T. Eary intensive behaviora treatment: Repication of the UCLA mode in a community setting. Journa of Deveopmenta & Behaviora Pediatrics 6; 7: S45-S55. 3. Achenbach TM. The Achenbach System of Empiricay Based Assessment (ASEBA): Deveopment, findings, theory and appications, Burington, University of Vermont Research Center for Chidren, Youth & Famiies 9. 4. Smith T, Groen AD, Wynn JW. Randomized tria of intensive eary intervention for chidren with pervasive deveopmenta Curr Pediatr Res 6 Voume Issue & 6
Soh/Migita/Takahashi/Shimatani/Hayashi/et a. disorder. American Journa on Menta Retardation ; 5: 69-85. 5. Mahoney G, Peraes F. Reationship-focused eary intervention with chidren with pervasive deveopmenta disorders and other disabiities: A comparative study. Journa of Deveopmenta & Behaviora Pediatrics 5; 6: 77-85. 6. Boxhi EH. Music therapy for the deveopmentay disabed, NY. Aspen Pubishers, Inc 985. 7. Thaut M. Rhythm, music and the brain: Scientific foundations and cinica appications, NY. Routedge 5. 8. Maone AB. The effects of ive music on the distress of pediatric patients receiving intravenous starts, venipunctures, injections and hee sticks. Journa of Music Therapy 996; 33: 9-33. 9. Kim J, Wigram T, God C. The effects of improvisationa music therapy on joint attention behaviors in autistic chidren: A randomized controed study. Journa of Autism and Deveopmenta Disorders 8; 38: 758-766.. Wigram T, God C. Music therapy in the assessment and treatment of autistic spectrum disorder: Cinica appication and research evidence. Chid: Care, Heath and Deveopment 6; 3: 535-54.. Oshima C, Itou N, Nishimoto K, et a. An accompaniment system for heaing emotions of patients with dementia who repeat stereotypica utterances: Toward usefu services for edery and peope with disabiities. 9th Internationa Conference on Smart Homes and Heath Teematics ; 679: 67-7.. Kurizuka Y, Miyake Y, Kobayashi Y. Waking support system based on musica exercise therapy. SICE 4 Annua Conference 4; 3: 693-697. 3. Giberg C. Hyperactivity, inattention and motor contro probems: prevaence, comorbidity and background factors. Foia Phoniatrica et Logopaedica 998; 5: 7-7. in postura contro and movement performance during goa-directed reaching in chidren with deveopmenta coordination disorder. Human Movement Science ; : 583-6.. Mari M, Castieo U, Marks D, et a. The reach-to-grasp movement in chidren with autism spectrum disorder: Phiosophica Transactions of the Roya Society B. Bioogica Sciences 3; 358: 393-43.. Cook J, Saygin AP, Swain R, et a. Reduced sensitivity to minimum-jerk bioogica motion in autism spectrum conditions. Neuropsychoogia 9; 37: 375-378.. Fash T, Hogan N. The coordination of arm movements: an experimentay confirmed mathematica mode. The Journa of Neuroscience 985; 5: 688-73. 3. Uno Y, Kawato M, Suzuki R. Formation and contro of optima trajectory in human mutijoint arm movement. Bioogica Cybernetics 989; 6: 89-. 4. Harris CM, Wopert DM. Signa-dependent noise determines motor panning. Nature 988; 394: 78-784. 5. Migita R, Shimatani K, Shibanoki T, et a. A markeress monitoring system for behaviora anaysis of infants using video images. Japanese Journa of Human Growth and Deveopment Research 4; 65: -7. Correspondence to: Zu Soh, Institution of Engineering, Hiroshima University, -4- Kagamiyama Higashi-Hiroshima, Hiroshima, Japan. Te: +8-8-44-5763 E-mai: sozu@bsys.hiroshima-u.ac.jp 4. Thema MP, Jan PP, Hay DA. Fine and gross motor abiity in maes with ADHD. Deveopmenta Medicine and Chid Neuroogy 3; 45: 55-535. 5. Yamamoto J, Kusumoto C. Deveopment and support for autistic spectrum disorder. Cognitive Studies 7; 4: 6-639. 6. Rogers SJ, Wiiams JHG. Imitation in autism: Findings and controversies, in imitation and the socia mind and typica deveopment, Say J. Rogers and Justin H. G. Wiiams eds., NY. Guiford 6: 77-39. 7. Boso M, Emanuee E, Minazzi V, et a. Effect of ong-term interactive music therapy on behavior profie and musica skis in young aduts with severe autism. The Journa of Aternative and Compementary Medicine 7; 3: 79-7. 8. Green D, Baird G, Barnett AL, et a. The severity and nature of motor impairment in Asperger's syndrome: A comparison with specific deveopmenta disorder of motor function. Journa of Chid Psychoogy and Psychiatry ; 43: 655-668. 9. Johnston LM, Burns YR, Brauer SG, et a. Differences 7 Curr Pediatr Res 6 Voume Issue &