A REVIEW ON PRINTED MUSIC RECOGNITION SYSTEM DEVELOPED IN INSTITUTE OF COMPUTER SCIENCE IAŞI

Similar documents
Primitive segmentation in old handwritten music scores

Optical Music Recognition: Staffline Detectionand Removal

Development of an Optical Music Recognizer (O.M.R.).

USING A GRAMMAR FOR A RELIABLE FULL SCORE RECOGNITION SYSTEM 1. Bertrand COUASNON Bernard RETIF 2. Irisa / Insa-Departement Informatique

Optical Music Recognition System Capable of Interpreting Brass Symbols Lisa Neale BSc Computer Science Major with Music Minor 2005/2006

GRAPH-BASED RHYTHM INTERPRETATION

Accepted Manuscript. A new Optical Music Recognition system based on Combined Neural Network. Cuihong Wen, Ana Rebelo, Jing Zhang, Jaime Cardoso

Towards the recognition of compound music notes in handwritten music scores

Optical music recognition: state-of-the-art and open issues

Ph.D Research Proposal: Coordinating Knowledge Within an Optical Music Recognition System

Department of Computer Science. Final Year Project Report

Music Representations

A COMPUTER VISION SYSTEM TO READ METER DISPLAYS

BUILDING A SYSTEM FOR WRITER IDENTIFICATION ON HANDWRITTEN MUSIC SCORES

2. Problem formulation

APPENDIX A: ERRATA TO SCORES OF THE PLAYER PIANO STUDIES

Machine Vision System for Color Sorting Wood Edge-Glued Panel Parts

VISUAL CONTENT BASED SEGMENTATION OF TALK & GAME SHOWS. O. Javed, S. Khan, Z. Rasheed, M.Shah. {ojaved, khan, zrasheed,

Symbol Classification Approach for OMR of Square Notation Manuscripts

TOWARD AN INTELLIGENT EDITOR FOR JAZZ MUSIC

Improving Performance in Neural Networks Using a Boosting Algorithm

Study Guide. Solutions to Selected Exercises. Foundations of Music and Musicianship with CD-ROM. 2nd Edition. David Damschroder

Efficient Processing the Braille Music Notation

The MUSCIMA++ Dataset for Handwritten Optical Music Recognition

Distortion Analysis Of Tamil Language Characters Recognition

Automated extraction of motivic patterns and application to the analysis of Debussy s Syrinx

An Empirical Study on Identification of Strokes and their Significance in Script Identification

CS229 Project Report Polyphonic Piano Transcription

Reducing False Positives in Video Shot Detection

Figure 2: Original and PAM modulated image. Figure 4: Original image.

An Integrated Music Chromaticism Model

Automatic Commercial Monitoring for TV Broadcasting Using Audio Fingerprinting

A Framework for Segmentation of Interview Videos

Music Radar: A Web-based Query by Humming System

Overview. Teacher s Manual and reproductions of student worksheets to support the following lesson objective:

xlsx AKM-16 - How to Read Key Maps - Advanced 1 For Music Educators and Others Who are Able to Read Traditional Notation

Automatically Creating Biomedical Bibliographic Records from Printed Volumes of Old Indexes

Color Image Compression Using Colorization Based On Coding Technique

APPLICATIONS OF DIGITAL IMAGE ENHANCEMENT TECHNIQUES FOR IMPROVED

Renotation from Optical Music Recognition

Computational Modelling of Harmony

Hidden Markov Model based dance recognition

OPTICAL MUSIC RECOGNITION WITH CONVOLUTIONAL SEQUENCE-TO-SEQUENCE MODELS

Orchestration notes on Assignment 2 (woodwinds)

MusicHand: A Handwritten Music Recognition System

AUTOMATIC MAPPING OF SCANNED SHEET MUSIC TO AUDIO RECORDINGS

Northeast High School AP Music Theory Summer Work Answer Sheet

ART OF FILM A WAY OF ARCHITECTURAL COMMUNICATION

Music Database Retrieval Based on Spectral Similarity

Smart Traffic Control System Using Image Processing

INTER GENRE SIMILARITY MODELLING FOR AUTOMATIC MUSIC GENRE CLASSIFICATION

Topics in Computer Music Instrument Identification. Ioanna Karydi

BREAKING ACCESSIBILITY BARRIERS Computational Intelligence in Music Processing for Blind People

Representing, comparing and evaluating of music files

Semi-supervised Musical Instrument Recognition

y POWER USER MUSIC PRODUCTION and PERFORMANCE With the MOTIF ES Mastering the Sample SLICE function

Preparing a Paper for Publication. Julie A. Longo, Technical Writer Sue Wainscott, STEM Librarian

A HIGHLY INTERACTIVE SYSTEM FOR PROCESSING LARGE VOLUMES OF ULTRASONIC TESTING DATA. H. L. Grothues, R. H. Peterson, D. R. Hamlin, K. s.

Getting Started. Connect green audio output of SpikerBox/SpikerShield using green cable to your headphones input on iphone/ipad.

A Review of Fundamentals

VivoSense. User Manual Galvanic Skin Response (GSR) Analysis Module. VivoSense, Inc. Newport Beach, CA, USA Tel. (858) , Fax.

German Lute Tablature Recognition

Adaptive Key Frame Selection for Efficient Video Coding

How to Read Just Enough Music Notation. to Get by in Pop Music

Chord Classification of an Audio Signal using Artificial Neural Network

UNIVERSAL SPATIAL UP-SCALER WITH NONLINEAR EDGE ENHANCEMENT

Music Representations

Lab P-6: Synthesis of Sinusoidal Signals A Music Illusion. A k cos.! k t C k / (1)

Singer Traits Identification using Deep Neural Network

Micro-DCI 53ML5100 Manual Loader

OBJECTIVE EVALUATION OF A MELODY EXTRACTOR FOR NORTH INDIAN CLASSICAL VOCAL PERFORMANCES

(12) United States Patent

A STATISTICAL VIEW ON THE EXPRESSIVE TIMING OF PIANO ROLLED CHORDS

Robert Alexandru Dobre, Cristian Negrescu

Student Performance Q&A: 2001 AP Music Theory Free-Response Questions

MUSIC THEORY CURRICULUM STANDARDS GRADES Students will sing, alone and with others, a varied repertoire of music.

Keys: identifying 'DO' Letter names can be determined using "Face" or "AceG"

Student Performance Q&A:

A Fast Alignment Scheme for Automatic OCR Evaluation of Books

Music Information Retrieval with Temporal Features and Timbre

Primo Theory. Level 5 Revised Edition. by Robert Centeno

1 Ver.mob Brief guide

jsymbolic 2: New Developments and Research Opportunities

* This configuration has been updated to a 64K memory with a 32K-32K logical core split.

SIMSSA DB: A Database for Computational Musicological Research

MUSC 133 Practice Materials Version 1.2

TechNote: MuraTool CA: 1 2/9/00. Figure 1: High contrast fringe ring mura on a microdisplay

PERCEPTUAL QUALITY OF H.264/AVC DEBLOCKING FILTER

Melodic Pattern Segmentation of Polyphonic Music as a Set Partitioning Problem

AP Music Theory Westhampton Beach High School Summer 2017 Review Sheet and Exercises

Instrument Recognition in Polyphonic Mixtures Using Spectral Envelopes

VISUAL INTERPRETATION OF ARCHITECTURAL FORM

ATSC Standard: Video Watermark Emission (A/335)

Perception-Based Musical Pattern Discovery

MUSIC scores are the main medium for transmitting music. In the past, the scores started being handwritten, later they

Automatic LP Digitalization Spring Group 6: Michael Sibley, Alexander Su, Daphne Tsatsoulis {msibley, ahs1,

SMART VEHICLE SCREENING SYSTEM USING ARTIFICIAL INTELLIGENCE METHODS

MATCHING MUSICAL THEMES BASED ON NOISY OCR AND OMR INPUT. Stefan Balke, Sanu Pulimootil Achankunju, Meinard Müller

Smearing Algorithm for Vehicle Parking Management System

1ms Column Parallel Vision System and It's Application of High Speed Target Tracking

Transcription:

BULETINUL INSTITUTULUI POLITEHNIC DIN IAŞI Publicat de Universitatea Tehnică Gheorghe Asachi din Iaşi Tomul LX (LXIV), Fasc., 24 SecŃia AUTOMATICĂ şi CALCULATOARE A REVIEW ON PRINTED MUSIC RECOGNITION SYSTEM DEVELOPED IN INSTITUTE OF COMPUTER SCIENCE IAŞI BY DAN GÂLEA, FLORIN ROTARU, SILVIU-IOAN BEJINARIU *, MIHAI BULEA 2, DAN MURGU 2, SIMONA PESCARU 2, VASILE APOPEI, MIHAELA MURGU 2 and IRINA RUSU 2 Institute of Computer Science, Romanian Academy, Iaşi Branch 2 Former Members of Institute of Computer Science, Romanian Academy, Iaşi Branch Received: February 2, 24 Accepted for publication: March 7, 24 Abstract. This paper presents recognizing complex printed music symbols techniques developed in our institute some time ago. The mentioned methods were part of a larger application for printed music scores recognition. Presented are the techniques for detection and removal of staff lines and measure bars, detection and recognition of note heads, stems and beams from a digitized image. Key words: musical symbols recognition, staff lines detection, symbol classification, musical information reconstruction. 2 Mathematics Subject Classification: 68T, 68U.. Introduction We started to design and implement the presented system at beginning of the last decade of the last century as part of collaboration with an Italian Society to develop software instruments for a music publishing house. Due the fact at that time very little documentation on the field was available, the optical music recognition (OMR) system was developed mainly using the customer * Corresponding author; e-mail: silviu.bejinariu@gmail.com

5 Dan Gâlea et al. specifications. At the time we published first papers on recognition of specific musical symbols (Bulea, 993), and on structural recognition of printed music (Rotaru & Bejinariu, 996), we could consult only reference (Sicard, 992) in the field. However because our two mentioned publications did not reflect the complexity of the whole OMR task we think it is worthwhile to describe those OMR methods less mentioned, presented or published. Also, it is interesting to compare the performance of our system with the ones of OMR architectures reported lately. In (Rebelo et al., 2) and (Rebelo et al., 22) presented are the main OMR papers published at the analysis time. The analysis leads the authors to the conclusion that, generally, the proposed OMR systems encompass four main stages: image preprocessing - several techniques as enhancement, binarization, noise removal, blurring, deskewing are applied to the music score to make the recognition process more robust and efficient. Staff line thickness and vertical line distance within the same staff are often computed, providing the information for relative size comparisons; recognition of musical symbols a process subdivided in three parts: (a) staff line detection and removal to obtain the musical symbols; (b) symbol primitive segmentation and (c) symbol recognition. The input in the third stage can be simple features, raw pixels and high-level features such as information on connected components or the orientation of the symbol. reconstruction of the musical information in order to build a logical description of musical notation - the symbol primitives are merged to form musical symbols. Graphical and syntactic rules are used to introduce context information to validate and solve ambiguities. construction of a musical notation model to be represented as a symbolic description of the musical sheet. The system output is a graphical music-publishing file, like MIDI or MusicXML. Presented are the states of the art for each of the four OMR system components. Another study (Dalitz et al., 28) presents a quantitative comparison of different algorithms for the removal of staff lines from music score images and suggests a new skeletonization based approach. The analysed algorithms are compared with respect to three defined error metrics. The robustness of each evaluated method is measured with respect to certain image defects. In (Fotinea et al., 2) proposed are two OMR approaches: the first one recognizes the musical symbols without removing the staff lines; the second one is based on staff lines removal. The first task is accomplished in three stages: (a) Preprocessing - including normalization of the music image, detection of staff lines and staff segmentation; (b) Recognition divided in three steps: recognition of cut-alone symbols; recognition of symbols containing at least one vertical line and recognition of remaining symbols; (c) Reasoning -

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 5 the recognition results of the previous two stages are combined and tuned using music theory rules. The second approach is done also in three steps: (a) Preprocessing including the following stages: evaluation of the dimensions of the music scores; isolation of separated staves; staff lines removal and isolation of connected components; (b) Isolated components classification - classification of standard-shaped symbols; classification of isolated notes and note groups; remaining components classification; (c) High level Reasoning. Fujinaga proposes in (Fujinaga, 996) an adaptive optical music recognition system where the database of the learned musical symbol classes is updated during the classification process. The adaptiveness is founded on two concepts: k-nn rule and genetic algorithms. Also, the thesis contains an interesting state of the art, starting with OMR systems realized at the beginning of 98 decade. A generalized framework (Bainbridge & Bell, 2) for OMR software is presented emphasizing key stages that must be solved: staff line identification, musical object location, musical feature classification and musical semantics. The authors describe the challenges posed by optical music recognition. A state in the art is included. Some other papers propose specific methods to solve certain tasks of an OMR system. For instance in (Vigliensoni et al., 23) is presented an image processing algorithm that extracts the position of bar lines on an input music score in order to deduce the number and position of measures on the page. In (Miyao, 22), a method to extract the five staff lines is proposed. The algorithm was tested and validated on musical sheets in which there are some inclinations, discontinuities, and curvatures. The author reports an extraction rate of 99.4% for 7 printed music scores. Carl Witt proposed in (Witt, 23) an approach to recognize music symbols using contour traces. The method has the following advantages: (a) solves the segmentation problem and the classification problem in a unified approach avoiding the error propagation from the segmentation stage; (b) make use of a search space that provides rich information, adequate for both detecting characters and parameterized shapes and (c) is able to deal with typical OMR challenges, such as font variations, touching symbols and broken symbols. Reed and Parker provide in (Reed & Parker, 996) and (Parker, 996) an overview of the techniques used by their system to perform tasks as: staff line detection, text segmentation, line detection, symbol recognition, note head recognition and semantic interpretation. Among these techniques are: template matching, Hough transform, line adjacency graphs, character profiles and graph grammars. In (Knopke & Byrd, 27) the authors propose a prototype system that uses multiple OMR recognizers to produce a composite document that is superior to the output of any one of the programs.

52 Dan Gâlea et al. All above mentioned papers report good or very good recognition rates for overall musical sheets or for specific groups of musical symbols. However, none of them seem to solve, as our system, very complex musical structures, as indicated in section 4 of the paper. 2. Optical Music Recognition (OMR) System The goal of this application is to recognize music symbols from the printed music documents. First, each page of music is digitized using a scanner. To extract symbols from the resulting binary image, the following steps are performed: Interactive staff separation. This step optional and it allows the user to clearly distinguish the boundaries between staffs and to properly assign music symbols to staffs for complex musical scores. Detection and removal of staff lines. The staff lines must be removed to obtain isolated musical symbols. This is completed using methods which do not cause distortions to the remaining symbols. Detection and recognition of musical symbols. Each object is framed and processed through a four level classification process, shown below. A pre-classifier was developed to accomplish two tasks, using projection analysis methods. The first task is to separate symbols that inadvertently touch or are connected (e.g. notes with sharps, naturals, flats). The second task approximates locations where notes exist. Structural and syntactic methods detect sub-patterns (note head, bar, stem, beam) for large complex symbols (all at level in Fig. ). A neural classifier (level 2 in Fig. ) is used to recognize small symbols (i.e. sharps, flats and pauses). Included in the decision process are a robust method for feature extraction and an algorithm for continuously adapting the training set from the newly obtained patterns (Bulea, 993). On the last level a specialized classifier analyses the objects rejected from the first levels, whole and half notes simple or in chords. The application includes mechanisms to classify note heads and beams. The localization of these symbols uses edge detection, thinning, digital distance computation, component labeling and reconstruction from incomplete border algorithms (Duda & Hart, 973; Meisel, 972; Rosenfeld & Kak, 976; Sicard, 992; Fu, 974; Pavlidis, 977; Banks, 99; Lindely, 99; Rotaru et al., 993; Rotaru, 998). The rest of the paper is structured in 4 sections. Section 3 describes the method for detection and removal of staff lines and measure bars. Section 4 presents structural and decisional methods to recognize music symbols. Section 5 discusses the results and evaluates the performance of the system.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 53 Fig. Classifier structure. 3. Detection and Removal of Staff Lines and Measure Bars One important step before symbol recognition is the detection and extraction of the staff lines. The symbol shape is more easily recognized without these lines. The staff line information, position of the lines, distance between lines, slope and the thickness of lines, is retained. All of this information is merged later, so that the total musical meaning of symbols and staff lines can be described. Some music symbols share segments with the staff lines. The removal of lines must be done in a method that keeps the connectivity of the symbols. This is a very difficult method to perfect and sacrifices had to be made with bass clef and whole notes located between staff lines. At a later step these symbols will be recognized by unifying two or three objects with special features that are located next to each other. The removal of staff lines is done in the following steps: a starting position is found by checking for 5 horizontal lines of the same thickness and equally spaced. This starting point is not necessary the left limit of the staff.

54 Dan Gâlea et al. from the start position a tracking and marking process of the lines to the left and to right is done. The tracking is performed in a common direction for all 5 lines of the staff. Intermediate points along the line, where the slope changes, are stored. The problem with printed music sheets and staff lines is that the lines are not perfectly straight. A straight line interpolation of a deformed printed line could result in an inaccurate placement of a symbol. For instance a note could be vertically misplaced, thus changing the tone of the note. This process of storing intermediate points and using the line segments between these points leads to a more accurate piecewise linear interpolation. The detection process continues regardless of gaps in the staff lines (caused by poor print quality or incorrect threshold to binarize the image). When a gap occurs in all five lines of the staff the process stops. the extraction of the five lines is completed by analyzing the locations where marked lines cross music symbols. The decision to cancel or not cancel the marked pixels is very similar to that used in thinning algorithms. Experience has shown that the algorithm to detect and cancel the staff lines performs the removal and preserves the shape of the majority of music symbols. The results of staff line and measure bar elimination are shown in Fig. 2 a (original image) and Fig. 2 b (processed image). Fig. 2 a Original image. After the staff lines elimination the measure bars removal is performed. The removal of the measure bars is required because music symbols are often connected to them. The detection of the measure bars is completed on a staff system level for each page. The first step is to detect the staff systems (staves) in the page. The detection process uses the separation line completed by the user in the interactive staff separation step. If a vertical line intersects the separation line the application associates the upper and the lower staffs to be in the same stave. The process is only completed on the beginning (left side) of the sheet music.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 55 A two-step process is used to validate the measure bar removal. First, the locations of well separated peaks from the X projection pixel histogram are determined. Second, a vertical line, identified from the first step, exceeding one or both five line staff limits, is removed only if the line intersects the appropriate staff separation boundary. The measure bars can not be confused with note stems, using this two-step process. Fig. 2 b Processed image. 4. Detection and Recognition of Musical Symbols The music symbols are identified by scanning the binary image of the score page. The scan is conducted line by line searching for a black pixel that can be a starting point for a contour tracking algorithm. From the detected pixel the symbol is localized using this tracking algorithm. A filling algorithm is then applied to change the pixels of the symbol to a different gray level to avoid detecting the same object twice. For each symbol the following information is stored: coordinates of the surrounding rectangle; coordinates of the starting point; area of the symbol; perimeter of the symbol. A major issue in music symbol recognition is to proper identify pitches (note head) and beam cross bars. Within this major issue is the proper identification of alterations (flats, naturals or sharps) which are connected to certain notes. This proper identification process is important because of the potential confusion between elements, such as: short beam cross bars improperly classified as pitch; pitches tied in chords creating a black pixel zone that can be misidentified as a beam cross bar.

56 Dan Gâlea et al. Structural recognition methods consider objects having a complex structure. Objects are analyzed and separated in basic shapes: notes, bars, beams and so on. The detection of filled notes (less than half value) and beam bars are solved using a set of thinning algorithms, segmentation (in order to detect regions which are situated at a certain distance to the background) and edge detection. 4.. Pitch and Chord Identification The pitch identification algorithm first identifies the local maximum zones (with pixels situated at the greatest distance from the background). The connected components of these zones are then labeled. After labeling the weight centers of the connected components are computed, according to certain parameters (staff lined thickness, distance between two lines). This operation is called focusing. The obtained centers allow the exploration from inside the analyzed symbol and the extraction of some shape characteristics. The centers with shape characteristics outside the accepted shape features space are eliminated. Pitches are detected in simple configuration (without connected notes) or in chords with connected adjacent pitches. An example of the segmentation, labeling and focusing processes is displayed in Fig. 3. Fig. 3 Pitch detection. The obtained results are compared with the note information provided by the pre-classifier, mentioned earlier. If the results do not match, more iterations of the segmentation and focusing processes are computed adjusting parameters (according to the error) until the results converge. For instance, if the pitch identification algorithm recognizes fewer simple notes than the preclassifier, the segmentation and focusing processes are computed again. The parameters are modified so that the local maximum area will be enlarged and more weight centers will be computed.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 57 After pitch detection the corresponding stems are identified, using a relatively simple tracking process. The level of the note head is established by identification of the closest staff line, or two adjacent staff lines, to the pitch center. The line equations were computed during the staff lines detection and centers coordinates during focusing. For each symbol classified as pitch the following information is computed and stored: position ( X, Y coordinates) level ( according to staff lines) stem location (to left or right) stem direction (up or down) positions of neighboring notes in a chord equivalent pitches (up, down) indicating the music duration. Symbols classified as pitches are divided in equivalency classes according to the stem they are connected to. For each class, an element is chosen to be analyzed for establishing the duration. This duration will be attributed to the whole equivalency class. In orchestral compositions there are portions of music which are very large and complicated as depicted in Fig. 4. The object can represent a complex musical structure with many notes tied to beams which form a large area of black pixels. In this case the segmentation algorithm requires a large amount of computing time and memory. A Y-projection method is applied to detect the beam area, for a very large and X-elongated object. This area is isolated and the segmentation process works only for the note area. Fig. 4 Complex musical structure. Fig. 5 shows two relevant views, the area of a unprocessed complicated symbol (top) and the processed image with the beam area isolated (lower left).

58 Dan Gâlea et al. Fig. 5 Beam segmentation of the musical structure illustrated by Fig. 4. 4.2. Identification of Beam Cross Bars The initially stages of the beam cross bar identification algorithms are used for digital distance computing and marking zones that contain pixels with distances larger than a dynamically established threshold. In beam cross bar recognition considerations had to be given to the fact that a logic beam has one or more bars which might fall into one of the following three situations: unconnected bars misconnected bars due to poor printing quality or incomplete staff line elimination one logic bar divided into components. These situations are depicted in the next figure: In the next stage of the recognition the marked symbols are analyzed using the following characteristics: X elongation Y elongation area/perimeter ratio surrounding rectangle dimensions ratio.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 59 Fig. 6 a 2 unconnected beam bars. Fig. 6 b 3 misconnected beam bars. Using this process the beam cross bars are identified (physically) and labeled. The labeled bars are then analyzed for a logic evaluation concerning the music duration of each pitch, using the following algorithm: for each main pitch in his equivalence class { for each of the stems to which is connected ( up-down and/or left-right ) { interest zone is established as a neighborhood of the stem interest zone is analyzed by a vertical scan along the stem for each beam cross bar encountered { the thickness within the interest zone is analyzed in order to establish the corresponding logic bars number if the analyzing bar is the first encountered then the bar label is associated to the pitch else if the bar label is the same with the precedent one we continue the loop else the bar takes the label from the precedent bar (the two bars are logically equivalent) } the musical duration for the associated note is established } the musical duration is completed for all equivalent pitches } The algorithm is designed to allow for the analysis and correct classification for each of the three previously presented situations. This remains true even the structure contains more than one logic beam, as in Fig. 7:

6 Dan Gâlea et al. Fig. 7 Structure with more than one logic beam. As illustrated in Figs. 4, 5 and 8 the structural analysis method can deal also with multi-staff complex structures. Fig. 8 Multi staff structure recognition.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 6 4.3. No Beam Connection Full Note Structure Analysis The third classifier from Fig. recognizes full note structures not connected to beams as depicted in Figs. 9 and. Fig. 9 No beam connection full note structure. A structural approach was used to analyze such musical objects. The note heads were detected using the method described in paragraph 4.. Then a horizontal projection of the object is performed and the minimum on the Y axis of the result is considered as the cut point to separe the head notes area from the stem area. From stem area Freeman code analysis there are detected the local minimum and maximum points. Considering some separability and symmetry conditions the stem repeater symbols can be detected as illustrated in Fig.. In order to establish the duration of pitches that are not connected to beams, the stem is separated from the note heads, as for repeaters, Fig. 9. Analysis of the Freeman chain-code obtained from the contour tracking algorithm, scaling and pattern matching methods are applied to classify quarter, eighth, sixteenth, and other durations notes.

62 Dan Gâlea et al. Fig. Note structure with repeater symbol recognition. 4.4. Half Notes Identification The third classifier from Fig. performs also steps to recognize half notes and half note chords (Fig. ). These steps are as follows: an acceptance criterion based on the dimensions of the surrounding rectangle is applied to accept/reject symbols that may be half notes; thinning of accepted symbols; reconstruction from incomplete border algorithms; stem localization and removal using X projection; detection of the remaining connected symbols and validation as half notes using Y projection analysis. Fig. Half notes processing.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 63 Reconstruction method from incomplete border algorithm labels the analyzed area pixels as follows: border pixels located outside the area to be completed; border pixels located inside the area to be completed; Blank for the others. For half notes this classification was done in four steps by four scans of the area between staff lines where staff line deletion could broke the head note. The area considered to include a broken half note is divided in four regions 2. Each region is scanned as follows: 3 4 Area : left right; up bottom; Area 2: right left; up bottom; Area 3: left right; bottom up; Area 4: right left; up bottom. The method labels the blank pixels according to following rules: Any blank with neighbors and is marked. Any blank with neighbors and blank is marked. Any blank with all neighbors is marked. Any blank with all neighbors is marked. The number of scans was experimentally established. The result is illustrated in Fig. 2. one step Fig. 2 Incomplete border reconstruction. The last step of processing a music page is to classify unrecognized objects by analyzing their localized area (for staccato or duration points) or connecting two or three symbols that are close together (bass clef parts, whole note parts, head note with interrupted stem).

64 Dan Gâlea et al. 5. Conclusions The program can recognize musical scores from a variety of different printing styles, sizes and symbol quality. The score recognition accuracy floats somewhere between 8% and 94% depending on the quality of the scanned pages. For good quality printed scores the recognition rate can reach 98%. After recognition, the page is corrected and completed using a specific music score editor. The recognition rate was evaluated in three ways: comparing the output OMR file with one resulted after editing completion of the score page. The comparison is made counting the recognized symbols but is not always relevant because simple structures can be wrong recognized while the complex structures could have o good recognition rate. More relevant is the time to complete the OMR result; the ratio (recognized symbols)/(number of sheet symbols after page editing); area percent computation of the recognized symbols. By the last evaluation the recognition rate is quite good because the level classifier provides very good results, and the area of those complex structures is large. However the time to edit from scratch a complex musical score is 6-7 times longer than completing the OMR result. For a medium quality image the recognition rate of each of the three classifiers is depicted below. The structural analyzer has a recognition rate of 94% for symbols connected in beam. For the neural classifier on level 2 the recognition rate is 9% and the recognition rate for simple note or connected in chords is 89%.

Bul. Inst. Polit. Iaşi, t. LX (LXIV), f., 24 65 REFERENCES Bainbridge D., Bell T., The Challenge of Optical Music Recognition. Computers and the Humanities, 35, 95 2, 2. Banks S., Signal Processing, Image Processing and Pattern Recognition. Prentice Hall, 99. Bulea M., Robust Recognition of Printed Musical Symbols Using Neural Networks. FSAI, 2, 2, 993. Dalitz C., Droettboom M., Pranzas B., Fujinaga I., A Comparative Study of Staff Removal Algorithms. IEEE TPAMI, 3, 5, May 28. Duda R.O., Hart P.E, Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 973. Fotinea S.-E., Giakoupis G., Liveris A., Bakamidis S., Carayannis G., An Optical Notation Recognition System for Printed Music Based on Template Matching and High Level Reasoning. The 6th Recherche d Informations Assistée par Ordinateur, Paris, 2. Fu K., Syntactic Methods in Pattern Recognition. Accademic Press, 974. Fujinaga I., Adaptive Optical Music Recognition. Ph. D. Thesis, McGill University Montréal, Canada, June 996. Knopke I., Byrd D., Towards MusicDiff: A Foundation for Improved Optical Music Recognition using Multiple Recognizers. Austrian Computer Society (OCG), 27. Lindely C.A, Practical Image Processing in C. John Wiley & Sons, 99. Meisel W.S., Computer-Oriented Approaches to Pattern Recognition. Academic Press, New York, 972. Miyao H., Stave Extraction for Printed Music Scores. In Yin H. et al. (Eds.), IDEAL, LNCS 242, 562 568, 22. Parker J.R., Algorithms for Image Processing and Computer Vision. John Wiley & Sons, First Edition, 996. Pavlidis T., Structural Pattern Recognition. Springer - Verlag, Berlin, Heidelberg, 977. Rebelo A., Capela G., Cardoso J.S., Optical Recognition of Music Symbols: a Comparative Study. Int. J. Document Anal. Recognit., 3, 9 3, 2. Rebelo A., Fujinaga I., Paszkiewicz F., Optical Music Recognition: State-of-the-Art and Open Issues. International Journal of Multimedia and Information Retrieval, March 22. Reed K.T., Parker J.R., Automatic Computer Recognition of Printed Music. In: International Conference on Pattern Recognition, 83 87, 996. Rosenfeld A., Kak A.C., Digital Image Processing. Academic Press, New York, 976. Rotaru F., Gâlea D., Bejinariu S., Apopei V., Murgu D., Pescaru S., Murgu M., Rusu I., Jitca D., Metode structurale şi sintactice în recunoaşterea simbolilor muzicali. Raport de cercetare I.I.T. Iaşi, Academia Română, 993. Rotaru F., Bejinariu S., Recognition of Printed Music. Advanced in Modelling & Analysis, B, 36,, 43 5, 996. Rotaru F., ContribuŃii la dezvoltarea metodelor de recunoaştere a formelor cu aplicańii în analiza imaginilor. Ph. D. Thesis, Iaşi, 998. Sicard E., An Efficient Method for the Recognition of Printed Music. Proceedings of the th IAPR International Conference on Pattern Recognition, Hague, Netherlands, 992.

66 Dan Gâlea et al. Vigliensoni G., Burlet G., Fujinaga I., Optical Measure Recognition in Common Music Notation. International Society for Music Information Retrieval, 23. Witt C., Optical Music Recognition Symbol Detection using Contour Traces. Bachelor Thesis, Institut fur Informatik der Freien Universitat Berlin, 23. SISTEM DE RECUNOAŞTERE AUTOMATĂ A PARTITURILOR MUZICALE (Rezumat) În cadrul Institutului de Informatică Teoretică Iaşi a fost dezvoltat un sistem de recunoaştere automată a partiturilor muzicale tipărite, componentă a unui sistem complex pentru editarea şi publicarea partiturilor muzicale. Sistemul de recunoaştere a fost proiectat în principal pe baza specificańiilor oferite de beneficiarul sistemului, într-o perioadă în care accesul la literatura de specialitate era foarte redus. Cu toate acestea, sistemul dezvoltat are performanńe comparabile cu cele raportate ulterior în literatura de specialitate. Sistemele de recunoaştere automată a partiturilor tipărite au o structură complexă ce include în general componente pentru: preprocesare; recunoaşterea simbolurilor (detecńia şi eliminarea liniilor de portativ, segmentarea simbolurilor şi recunoaşterea propriu-zisă); reconstrucńia informańiei muzicale (validare şi eliminarea ambiguităńilor); generarea descrierii simbolice a informańiei. În cadrul acestui articol sunt detaliate componentele pentru detectarea şi eliminarea liniilor de portativ şi a barelor de măsură, detecńia şi recunoaşterea simbolurilor de tip cap de notă şi respectiv a structurilor complexe de note legate în beam. Prima secńiune conńine o trecere în revistă a sistemelor de recunoaştere a partiturilor, menńionate în literatura de specialitate. Cea de a doua secńiune prezintă structura generală a sistemului dezvoltat. În cea de a treia secńiune a lucrării este prezentată metoda propusă pentru detecńia şi eliminarea liniilor de portativ şi a barelor de măsură. A patra secńiune a lucrării detaliază metodele utilizate pentru recunoaşterea capurilor de notă simple sau legate în structuri de tip chord, identificarea capurilor de notă conectate în structuri complexe de tip beam, inclusiv multi-portativ, precum şi a notelor întregi respectiv a doimilor. În ultima secńiune a lucrării sunt prezentate evaluări ale rezultatelor obńinute prin aplicarea sistemului de recunoaştere realizat. Pentru o imagine de calitate medie, ratele de recunoaştere sunt de 94% pentru simbolurile conectate în structuri de tip beam, 9% pentru simbolurile simple analizate folosind clasificatorul neuronal, respectiv 89% pentru notele simple sau legate în structuri de tip chord.