I Throw Iching Powder a Tulips Richard P. Gabriel IBM Research rpg@dreamsongs.com Absrac Programming comes in many shapes & sizes. Caegories and Subjec Descripors D.2.9 [Sofware process models] General Terms Experimenaion Keywords Agile; science; programming; naural language generaion I wan o remind you of somehing simple: Programming and sofware engineering are no he same hings. Neiher is programming he same as algorihm design. We ve angled he several noions of programming if we ry we can unweave hem, bu someimes we push on oo quickly / ge confused. William Griswold venured his definiion of sofware engineering: The pracice of consrucing sofware o saisfy all sakeholder requiremens so as o maximize value William Griswold, personal communicaion, 2013 Programming is more fundamenal. Venuring a guess, I would define programming as designing a se of mechanisms o enable some device (very broadly consrued) o do somehing i normally could no using mechanisms i already has. Thinking abou compuers is he simple case: programming is puing ogeher insrucions in a programming language ha is easy for people o undersand which cause he underlying mechanisms of he compuer including is physical and elecrical componens o realize he purpose of he Permission o make digial or hard copies of all or par of his work for personal or classroom use is graned wihou fee provided ha copies are no made or disribued for profi or commercial advanage and ha copies bear his noice and he full ciaion on he firs page. Copyrighs for componens of his work owned by ohers han he auhor mus be honored. Absracing wih credi is permied. To copy oherwise, or republish, o pos on servers or o redisribue o liss, requires prior specific permission and/or a fee. Reques permissions from permissions@acm.org. Onward! 14, Ocober 20 24, 2014, Porland, Oregon, USA. Copyrigh is held by he auhor. Publicaion righs licensed o ACM. ACM 978-1-4503-2585-1/14/10 $15.00. hp://dx.doi.org/10.1145/2661136.2661155 1 program. Bu i also works for physical devices, biological sysems, and people oo. For example, when we each a child o add, we are creaing a program ha builds on he child s exising abiliy o coun on fingers. To he child he noion of adding is novel, bu perhaps couning on fingers is no. A firs, addiion is a program; laer i is an abiliy. When we describe how o drive from one place o anoher, ha s a program ha uses he driver s abiliy o undersand direcions, o drive, and o recognize ellales o ge ha person from one place o anoher. When people ry o pu ogeher large sofware sysems large enough ha eams are needed and dangerous enough ha safey is crucial hey apply engineering echniques (as bes hey can) o he projec. Tha s he sar of sofware engineering. When people wonder wheher he program hey have devised really will achieve is desired purpose using he underlying mechanisms of he compuer, ha s he sar of he heory of compuaion and algorihm design. In recen years he conflic of more radiional sofware engineering approaches and agile have made a muddle of he concep of programming o a degree where someimes only sofware engineering is considered programming. I hink. Sofware engineering is no wha I do when I program. I am programming. I wrie sofware as par of doing science. I use sofware as a machine or insrumen o explore how he mind / brain migh work. No when he mind s hinking ha s old AI. I mean when people are creaing. Sofware engineering is for producing somehing ha someone can describe, eiher using specificaions (he oldfashioned way) or a produc backlog (or some oher suchlike hing in an agile seing). I don do anyhing like ha. And sofware engineering is abou producing sofware in a group. The way we each programming makes everyhing sofware engineering. Pundis saemens are deceiving: Programmers mediae beween he negoiaed and uncerain ruhs of business and he crisp, uncompromising domain of bis and byes and higher consruced ypes. Kevlin Henney, 97 Things Every Programmer Should Know, 2010 [1]
This looks correc, innocen even. When I program I indeed worry wheher he programs I wrie will do somehing like wha I inend; bu here are no ruhs of business in sigh. The mos progressive eachers reinforce his and oher sereoypes, like his programming assignmen: Develop a funcion ha when given an iniial amoun of money (called he principal), a simple annual ineres rae, and a number of monhs will compue he balance a he end of ha ime. Assume ha no addiional deposis or wihdrawals are made and ha a monh is 1/12 of a year. Toal ineres is he produc of he principal, he annual ineres rae expressed as a decimal, and he number of years. Felleisen e al, How To Design Programs [3] A business-relaed problem; a problem posed o he suden by an ousider; a problem. Everyhing ha he program should accomplish is spelled ou, by an exper insrucor or cusomer of course i is solvable, of course i is achievable, of course all hinking is reduced o wheher he program achieves is inended purpose using he underlying mechanisms of he compuer a hand. Programming happens, bu in a limied conex. A very well-respeced professor venured his opinion o me: I hink he bigges misake we make wih he saring poin for undergraduae educaion is ha we inroduce programming a all. The righ saring poin, IMHO, is requiremens and specificaion ogeher wih he associaed mahemaics ha hey require. anonymous, ruminaing on a firs course Problem solving; being old wha o solve, wha goals o achieve. This is hammered ino sudens and job seekers who are asked repeaedly o solve problems (homework / job inerviews). The idea of a problem doesn necessarily encompass he ceriude of soluion, bu as augh in he conex of programming, he uninended implicaion is ha a problem is a puzzle, and puzzles have soluions. A puzzle is a es of ingenuiy. My characerizaion of why I program building a sofware machine o explore naure and creae a heory migh remind you of Peer Naur s Programming as Theory Building [2]. Naur is alking abou a relaed bu differen ask he creaion of sofware as he final goal: he primary aim of programming is o have he programmers build a heory of he way he maers a hand may be suppored by he execuion of a program. Peer Naur [2] 2 The maers a hand are roughly he suff in he real world ha he program being buil needs o handle, and he heory in quesion is a concepual framework for undersanding he means by which sofware achieves ha handling. In my case, he heory is a scienific heory o be discovered, and he sofware is an insrumen o help discover / forge ha heory, which he sofware does by reacing o or revealing somehing abou he maerial agency of he world. The resuling sofware may or may no be ineresing by iself. My model of doing science wih sofware can be used o discover he Naurish heories programmers build o creae sofware, and i migh happen ha he sofware creaed in my model is he sofware he programmers evenually creae. Produciviy and value are essenials for business programming. Jeff Suherland wroe his abou Scrum: Scrum is a simple framework used o organize eams and ge work done more producively wih higher qualiy. I allows eams o choose he amoun of work o be done and decide how bes o do i, hereby providing a more enjoyable and producive working environmen. Scrum focuses on prioriizing work based on business value, improving he usefulness of wha is delivered, and increasing revenue, paricularly early revenue. Jeff Suherland, A Brief Inroducion o Scrum, 2007 [4] Noice ha he assumed conex is business, creaing value for a cusomer who drives requiremens and judges accepabiliy, and programming while consuming cash slowly and producing revenues quickly. Afer being augh (implicily) ha programming begins when someone (a eacher) ells you o sar, and ha he goal / opic / domain / problem for programming is given (by ha eacher), working as par of someone else s machine is no foreign. Suherland coninues: Designed o adap o changing requiremens during he developmen process a shor, regular inervals, Scrum allows eams o prioriize cusomer requiremens and adap he work produc in real ime o cusomer needs. By doing his, Scrum provides wha he cusomer wans a he ime of delivery (improving cusomer saisfacion) while eliminaing wase (work ha is no highly valued by he cusomer). Jeff Suherland, A Brief Inroducion o Scrum, 2007 [4] Agile s conribuion was o urn on is head he following premise of earlier sofware engineering mehodologies: change is expensive and needs o be avoided. Or a leas limied o he earlies possible pars of requiremens gahering and design. The idea is ha making a change o a design is cheaper han making a change o an implemenaion. Seve McConnell ells i his way:
In he wors case, reworking a sofware requiremens problem once he sofware is in operaion ypically coss 50 o 200 imes wha i would ake o rework he problem in he requiremens sage (Boehm and Papaccio 1988). I s easy o undersand why. A 1-senence requiremen can expand ino 5 pages of design diagrams, hen ino 500 lines of code, 15 pages of user documenaion, and a few dozen es cases. I s cheaper o correc an error in ha 1-senence requiremen a requiremens ime han i is afer design, code, user documenaion, and es cases have been wrien o i. Seve McConnell, 1996 [5] As if correcing ha 1-senence requiremen were ha easy. Jus change some of he words, righ? I s easy, for example, o change plans for a vacaion from driving o Bisbee, Arizona, o flying o he French Riviera jus change he modes of ransporaion and some hoels in he plan bu he cos of he change will hi hard laer. This sor of flawed hinking is super-easy o fall prey o while sharpening one s gullibiliy for hinking in sereoypes. If i s easy o change a bad requiremen o a good one, hink how easy i is o change a good requiremen o bad. Well, all one need do is expand he 1-senence o 5 pages of design diagrams and hink abou hem; or hen ino 500 lines of code and hink abou hem; or ino 15 pages of user documenaion and hen a few dozen es cases and hink abou hem ha s when you see he problem. Making he change is easy while knowing he change is smar is hard. This is where agile comes in. The bug, hey claim, is ha as long as he code being produced isn running in a way ha he cusomer can observe, errors in requiremens can persis longer han need be because he cusomer is unable o observe and hen inervene. They say Our highes prioriy is o saisfy he cusomer hrough early and coninuous delivery of valuable sofware. Agile Manifeso Principles [6] And his enables hem o Welcome changing requiremens, even lae in developmen. Agile processes harness change for he cusomer s compeiive advanage. Agile Manifeso Principles [6] The differences beween up fron planning and agile approaches seem similar o he differences in approaches o ar: experimenal versus concepual. To wi: for experimenal ariss: planning a paining is unimporan. The subjec seleced migh be simply a convenien objec of sudy, and frequenly he aris reurns o work on a moif he has used in he pas. Some experimenal painers begin 3 wihou a specific subjec in mind, preferring insead o le he subjec emerge as hey work. Experimenal painers rarely make elaborae preparaory skeches. Their mos imporan decisions are made during he working sage. The aris ypically alernaes beween applying pain and examining he emerging image; a each poin, how he develops he image depends on his reacion o wha he sees. Galenson, Old Masers and Young Geniuses [7] For concepual ariss: planning is he mos imporan sage. Before he begins working, he concepual aris wans o have a clear vision eiher of he compleed work or of he process ha will produce i. Concepual ariss consequenly ofen make deailed preparaory skeches or oher plans for a paining. Wih he difficul decisions already made in he planning sage, working and sopping are sraighforward. The aris execues he plan and sops when he has compleed i....exreme praciioners...make all he decisions for a work before beginning i. I is unclear, however, if his is lierally possible. There are ariss who came close o i, and perhaps achieved i, during he 1960s, by making plans for heir work and having hese plans execued by ohers. Galenson, Old Masers and Young Geniuses [7] Picasso was a concepual aris (mosly), and he did Big Design Up Fron. If here was a key 1-senence requiremen ha made possible his maserpiece, Les Demoiselles d Avignon [8], i ook Picasso more han 400 sudies and skeches (a record for arisic preparaion) o ge i righ. Bu as always, boh sofware engineering camps are vassals hey are handed problems, given direcion, and paid for piecework. In general. None of his is new hinking. And i s no really a he hear of he maer ha s engaging me. Righ now I am using programming very differenly from wha he sofware engineering approaches assume and celebrae, and also from wha eachers of programming prepare sudens for. I program o explore scienific quesions. Wha I produce are insrumens ha help me peer ino he unknown. I don work on puzzles bu on myseries. I don have cusomers; neiher requiremens; nor specificaions; nor es cases (really); nor design issues of he same sors as sofware engineers; here are no roadmaps; everyhing is a prooype and also an endproduc; I don know wheher he nex hing I ry will work, can work, should work; and I am profoundly disappoined when my programs fail o surprise me. If radiional sofware engineering and agile define wo poins on a specrum, wha
I do is as far from agile as agile is from radiional sofware engineering, wih agile in he middle beween my spo and radiional SE. This is how I see i. My nex ask is o show you wha I do and le you judge wheher i depics programming differenly from how sofware engineers and algorihm designers see i differenly from how you see i. I believe programming is a fundamenal ool of discovery and creaiviy which has been harnessed o serve he needs of indusry, capialism, as well as he greaer good ha he boring pars of programming are immensely valuable, bu also ha hose boring pars conain islands of programming like he coninen I live on. I am working on a naural language generaion (NLG) sysem as par of a DARPA projec. The specific hrus of my work is o ake a emplae of a couner-message and ailor i for he audience ha is, o he person or people o whom i will be sen. This Templae Reviser (as I originally called i i s called InkWell now) is inended o be a precursor o a fullblown naural language generaion (NLG) sysem. I worked on an NLG sysem for my PhD back in he lae 1970s, and his can be viewed as a maured sequel. When I worked on NLG back hen, I had an ineres in wriing and creaiviy bu no a lo of experience or educaion. Since hen I ve worked a bi on undersanding creaiviy, bu more imporanly I go an MFA in Creaive Wriing, namely in poery. I ve published a small book of poems, and have been wriing a poem a day for foureen years now. I ve wrien a novel (unpublished), and have four oher published books. Wih ha educaion and experience I ve come o realize ha wriing (creaively) involves a wide variey of implici influences and conribuing facors influences and facors ha deermine word choice, phrasing, and srucure. The plain meanings of words can ell one sory, and oher sories can be old by he connoaions of hose words, linkages beween ideas and images can be made wih sound he so-called music of he words and secondary and eriary srucures can be esablished by using hese and oher wriing craf elemens. The DARPA projec is called Social Media in Sraegic Communicaion (SMISC). One of is saemens of work: Take a emplae of a couner-message and ailor i for he audience. This goal is he exen of my requiremens and my only ineracion wih he cusomer. I inerac wih researchers hrough conversaions, email, research papers, and meeings, bu he purpose of hose ineracions is o ge ideas, repor on findings, and o gaher encouragemen. In no way do hese ineracions seem like insrucions, direcion, or orders. I believe ha hose sors of hings pop up in my work, bu from a very differen source, and for very differen purposes han for 4 sofware delivery o businesses. This is no a puzzle because here is no correc answer nor is anyone around o declare my program s couner-messages accepable. Commercial sofware is generally no exciing sofware. I rarely breaks new ground; if anyhing is difficul i s difficul because is algorihms migh be hard or performance is elusive or because he righ kinds of daa srucures are hard o pin down. Of course here are excepions. In many cases hese difficulies are hidden by frameworks, middleware, libraries, and he like. And hough wha he sofware does migh be boring, is design and consrucion are likely no, and here is remendous pleasure in designing and building somehing of value and beauy. Bu rarely is he consrucion of commercial sofware a grand challenge someimes i is, bu no frequenly. This realiy makes he ask of creaing commercial sofware mosly a maer of geing he deails he way he cusomer likes. Admirable, bu no my game. Consider: Bu merely exending knowledge a sep furher is no developing science. Breeding homing pigeons ha could cover a given space wih ever increasing rapidiy did no give us he laws of elegraphy, nor did breeding faser horses bring us he seam locomoive. Edward J. v. K. Menge [9] I do science. The scenario I work from is his: some group is engaged in a persuasion campaign ha deserves o be hwared his can be a phishing aack ( your accoun has been compromised; please re-ener your criical informaion here ), a proes ha could become dangerous ( WBC will picke he sodomie whorehouse and dog kennel masquerading as S. Agnes Caholic Church in religious proes and warning ), or erroris ploing. Through monioring social media, he organizers of he persuasion campaign and heir social neworks are idenified. Analysis produces he maerial required o plan a couner-messaging campaign. My program delivers ha campaign, using he maerials gahered. Wha makes his hard is ha i isn likely ha elling he persuaders don do ha is going o work, nor is i clear ha hose being argeed will respond more o my messages han o he persuaders. Wha needs o be undersood are he moivaions, he objecives, he incenives, and he syles of communicaion likely o work as dissuasion. I need o figure ou a program ha will generae messages ha include influences designed o dissuade and ha considers personal characerisics of he message recipiens. For example an influence migh be a bias oward choosing words wih cheerful connoaions or using Biblical rhyhms o im-
ply moral auhoriy. A personal characerisic could be a se of perceived needs and aiudes. My approach is o develop a se of emplaes bu no he boring kind you migh imagine figure ou how o selec he righ ones, figure ou how o compose hem eleganly, and deermine how o une hem for he audience. One of he primary ways o appeal o an audience is o exhibi a paricular se of personaliy rais. The group I work wih in he lab uses he so-called Big Five personaliy rais [10] [11] wih a good dose of oher personaliy faces and values. Big Five is a consolidaion of approaches o assessing personaliy based on examining exs people use o describe hemselves. For example, if a person says: I am he life of he pary I don mind being he cener of aenion I feel comforable around people I sar conversaions I alk o a lo of differen people a paries his is evidence ha he person is exravered. Each of he five rais represens a specrum wih he following endpoins and definiions: Openness: invenive/curious vs. consisen/cauious. Openness reflecs he degree of inellecual curiosiy, creaiviy, and a preference for novely and variey. Conscieniousness: efficien/organized vs. easy-going/careless. Conscieniousness reflecs a preference for organizaion, dependabiliy, discipline, duy, and achievemen planned insead of sponaneous. Exraversion: ougoing/energeic vs. soliary/reserved. Exraversion represens energy, posiive emoions, urgency, asseriveness, sociabiliy, and a endency o seek simulaion. Talkaiveness. Agreeableness: friendly/compassionae vs. analyical/deached. Agreeableness is compassion and cooperaion raher han suspicion and anagonism; i represens a rusing and helpful approach. Neuroicism: sensiive/nervous vs. secure/confiden. Neuroicism is he endency o experience unpleasan emoions easily (anger, anxiey, depression, and vulnerabiliy); i refers o emoional sabiliy and impulse conrol. [11] Personaliy rais are deermined by a sor of simple linguisic analysis o wha degree do you agree wih he following saemens? Bu researchers in my group have aken ha much furher. Raher han looking a responses o direced quesions, he analysis looks a exs people wrie. This work is based on wriing samples from voluneers who also have aken personaliy ess, and hen machine learning was used o esablish a funcion ha akes ex and produces a judgmen abou personaliy. This is prey clever work ha I 5 didn do, bu I needed o undersand i well enough o embed i in my NLG program. The process works by examining he words in a wriing sample, and couning he numbers of words in each of 68 caegories ([Figure 1] on he nex page). This approach is based on he work of James Pennebaker called Linguisic Inquiry and Word Coun (LIWC pronounced luke ) [12]. This yields a vecor of percenages for each caegory here is a corresponding percenage of words in he documen ha fall ino ha caegory. These couns are deermined by a dicionary ha maps words o caegories. For example, he word agony maps o caegories 12, 16, and 19, which are Affec, Negaive affec, and Sadness, resp. Using hese probabiliies, i s possible o compue an esimae of he wrier s Big-Five personaliy rais, using he work of Tal Yarkoni [13]. Yarkoni found correlaions beween LIWC scores and he Big Five rais, and hese correlaions are expressed as a simple linear combinaion of he probabiliies ha LIWC compues. Here s an example of all his. Suppose his is a ex we wan o analyze: Reading Gabriel s essays is pure agony. The LIWC scores for his are as follows: Caegory Coun Percenage of all Words Sadness (19) 1 16.7% Cogniion (20) 1 16.7% Percepion (27) 1 16.7% Cerainy (26) 1 16.7% Seeing (28) 1 16.7% Affec (12) 1 16.7% Presen (39) 1 16.7% Negaive Emoion (16) 1 16.7% This of course is oo small a sample for accuracy he LIWC dicionary recognizes only hese words: reading, is, agony bu his is jus an example of how he analysis works. Applying Yarkoni s coefficiens we ge he following Big- Five analysis: Trai Value Agreeableness -4.2% Conscieniousness -16.34% Exraversion Neuroicism 9.90% Openness -8.51% These values should be inerpreed like his: The sign says wheher he rai is eviden (+) or is opposie is (-); he mag-
All pronouns Numbers Cause@Causaion Social Space Leisure Sympoms & sensaions 1s person singular Affec Insigh Communicaion Up Home Sexual 1s person plural Posiive affec Discrepancy Reference o ohers Down Spor/exercise Eaing/drinking Toal 1s person Posiive feelings Inhibiion Friends Inclusion TV/movies Sleeping/dreaming Toal 2nd person Opimism Tenaiveness Family Exclusion Music Grooming Toal 3rd person Negaive affec Cerainy Humans Moion Money Swear words Negaions anxiey Sensaion/percepion Time Occupaion Meaphysical Non-fluencies Assens Anger Seeing Pas School Religion Fillers Aricles Sadness Hearing Presen Job Deah Preposiions Cogniion Touching Fuure Achievemen Physical saes/facors Figure 1 niude ranges from 0% o 100%, which represens he range of heoreically possible values. Exraversion is lef blank because here were no Yarkoni coefficiens associaed wih i in he se of non-zero elemens of he LIWC vecor meaning here is no evidence. A loud bray may be heard almos wo miles away. radiional The Agile Manifeso when I saw i he firs ime I laughed like a jackass. I sill chuckle and i s been years: Individuals and ineracions over processes and ools Working sofware over comprehensive documenaion Cusomer collaboraion over conrac negoiaion Responding o change over following a plan no because i s wrong, bu because I always have he feeling ha he auhors of his lis believed hey d aken he righhand saemens and saed heir opposies as heir favored values ha hey hough, for example, ha working sofware is an opposie of comprehensive documenaion, and ha by doing so hey have succeeded in making a se of ourageous (bu rue) saemens. (I love agile developmen, so don ge he wrong impression.) Take a look a my aphorisms for programmers doing science (immediaely before he references). Here s wha I mean, example-wise. The subile of Ken Beck s exreme programming book is Embrace Change [14]. This and he las bulle above make i seem like embracing change is a radical new idea, and as compleely differen from design up fron as you can ge. Bu embracing change is he midpoin of he range. One endpoin is o fear change so much you plan forever; he midpoin is o embrace change; and he opposie endpoin is o injec change / sugges i / insis on i. Oher dimensions migh exis. In my world of programming I use he following manifeso: Naure over individuals and ineracions Insighs over working sofware Problem engagemen over cusomer collaboraion Grappling wih mysery over responding o change 6 Le s look a one of hese for a minue. Wha does insighs over working sofware mean? To undersand how his LIWC / Big Five / Yarkoni hing works, I decided o program i up myself. Moreover, I needed code for his in he inner loop for he NLG sysem I was going o wrie. Afer his exercise I had working sofware righ in fron of me. My resuls replicaed all he published resuls I could find as well as he resuls from my group s Java implemenaion. While reading he Yarkoni paper I noiced his saemen: The resuls converge wih oher recen findings suggesing ha, conrary o popular wisdom, people do no presen hemselves in an idealized and overly posiive way online, and mainain online ideniies ha reflec he way hey genuinely see hemselves and are seen by ohers. Yarkoni [13] I slighly misook his o be saying ha people canno very easily hide heir personaliies when hey wrie. The work of Pennebaker seemed predicaed on his as well. And earlier in he Yarkoni paper I read, previous sudies have found sysemaic associaions beween personaliy and individual differences in word use [13]. A he op of he nex page is a se of Big-Five measuremens of six corpora. The ones inside he red oulines refer o hings I have wrien; looking from lef o righ, hey are my book Paerns of Sofware, a collecion of abou 5,000 of my poems (ha s no a ypo five housand), my book Wriers Workshops and he Work of Making Things, and an unpublished novel I wroe ( Tradiional Salvaion ) [Figure 2]. Tha is, essays, poems, nonficion, and ficion. The nex wo are Leaves of Grass by Wal Whiman and he colleced sories of Ernes Hemingway. One hing ha became clear o me looking a his char is ha I don correlae well wih myself. The red oulines are around hings I wroe each a differen genre. The righhand figure on he nex page ([Figure 3]) is grouped according o genre a bi. In he red oulines are my poems and Whiman s; no in oulines are my novel and Hemingway s sories. Now hings look correlaed beer bu by genre and no by person. I decided o do an experimen: wha if insead of he Big
20 15 Whiman Hemingway 20 15 Poery: Gabriel & Whiman 10 10 5 Gabriel 5 0 0-5 -5-10 -10 Ficion: Gabriel & Hemingway -15-15 Conscieniousness Exraversion Openness Agreeablness Neuroicism Conscieniousness Exraversion Openness Figure 2 Figure 3 Agreeablness Neuroicism Five personaliy rais, I posulaed hree genre-based rais: Poery, Ficion, and Nonficion. And like he work my group did, wha if I used machine learning o discover coefficiens ha would map LIWC scores o genres? My firs aemp was a failure. I ook a se of raining corpora, labeled hem wih binary judgmens poery, ficion, or nonficion, exclusively and used a simple machine learning algorihm o develop classifiers for each genre. The machine learning algorihm did no converge. If I looked a i as a sofware engineering problem, I would have explored wheher I had coded he machine learning algorihm correcly, or wheher perhaps I had no applied i correcly. I had been using his algorihm and my implemenaion of i for abou en years, so ha didn seem like a good approach. The reason for divergence was ha i was foolish o hink a piece of lieraure is purely poery or any of he oher genres. Wha abou narraive poery, lyrical ficion? I hen creaed a raining arge where each corpus was labeled as a mixure of genres [Figure 4]. Wih his he machine learning algorihm converged. When I esed i, hough, i was a lile off. I acked on wo decision rees one for he main genre and anoher for a hing I called a mixin genre, which is a kind of secondary characerisic. For example, Shakespeare s Hamle is classified by my classifier as Ficion wih a Poery mixin no a bad resul. Here are he wo decision rees one for main genre: -5.0 P (poery) -5.0 NF (nonficion) oherwise (ficion) and he oher for mixin genre: -10.0 P < -5.0 (poery mixin) -10.0 NF < -5.0 (nonficion mixin) 35.0 F (ficion mixin) P, NF, and F are he oupus of he hree learned sensors. The resuls of he classifier on a number of corpora are in [Figure 5]. (The parenheical judgmens are wha hose works are considered o be in lierary circles.) The sysem correcly idenifies genre more han 80% of he ime, and in he cases where i isn compleely righ, i s half-righ 37% of he ime. The predicae shows ha ficion is no special, and analyzing wha he genre sensors look for, one can idenify he characerisics ha deermine genre. This work also shows ha he LIWC insrumen probably mixes a variey of deeced signals, and so i is no a pure senimen insrumen. Agile goes half way wr change: from resis change o welcome change wha abou injec change? By creaing opporuniies for / making changes, a scienis explores, hen discovers, and laer undersands. The world was prey saisfied wih Newonian relaiviy for a long ime, bu physiciss kep monkeying wih he ideas. The genre deecor shows his. I had working code ha was perfecly fine, and my employer doesn really care abou lieraure and creaive wriing. Neverheless, my genre exploraion made my group quesion and explore he naure of he (compuaional) insrumens hey were building along wih he science behind i. Solid ground ruh is hard o come by because in mos cases i comes from sudies of undergraduaes and Mechanical Turkers. This implies ha i is usually a bad plan o si pa on he curren (scienific) undersanding and he code ha realizes ha undersanding s implicaions. A scienific pursui has no boss or eacher, aside from naure or realiy a creaure no ineresed in collaboraion. If you lock yourself away wih heory and ruminaion, you will dig yourself a hole wih you always a he boom, because he mind can so easily work on pure hough-suff. The mind needs somehing o play wih, and he more visceral ha play, he beer. Even wriing hings down and saring a hem, or ossing equaions or expressions on he chalk board and erasing / revising is needed. Sofware is a machine scieniss dream up o explore naure. Usually sofware is a way o explore daa associaed wih scienific research. Gaher a pile of daa and hen analyze 7
Corpus Poery Ficion NonFicion Poemsrpg (P) 85.0-10.0-10.0 Leaves of Grass (P) 95.0-30.0-50.0 Tradiional Salvaion (F) -10.0 80.0-25.0 Hemingway (F) -10.0 95.0-75.0 Paerns Of Sofware (NF) -35.0-5.0 95.0 Wriers Workshop (NF) -10.0-2.0 90.0 Faulkner (F) -5.0 95.0-65.0 Ulysses (F) -5.0 90.0-15.0 Emily Dickinson (P) 95.0-25.0-80.0 Unabomber (NF) -70.0-50.0 85.0 Wizard of Oz (F) -25.0 85.0-35.0 Call Of The Wild (F) -12.0 87.0-55.0 Huckleberry Finn (F) -5.0 45.0-40.0 Meamorphosis (F) -25.0 70.0-35.0 Origin Of Species (NF) -80.0-10.0 75.0 Figure 4 i. In he siuaions I m alking abou, he sofware iself is a direc pah ino he maerial being sudied his following he physicis urned philosopher, Andrew Pickering [16], who says ha science is a process of finding a sable poin beween a concepual framework (a heory) and he maerial agency of he world as revealed hrough machines or insrumens. Scieniss build machines o explore realiy, and wha he machine does or repors is inerpreed according o he concepual framework. When he machine repors somehing unexpeced or conradicory (a resisance in Pickering s erminology), he concepual framework is adjused along wih he machine (usually) (each adjusmen is called an accommodaion ), and more observaions are made. Evenually, he concepual framework, he machine, and he resisances sele down, and a fac is manufacured. In his sor of scienific exploraion, he sofware will alk o you. Bu I don mean ha i s alking abou iself he code I wrie is an inermediary beween my research hinking and he par of he world I m looking a. I s like a elescope from he early days of science or he Large Hadron Collider hese days. I don accep working sofware / I keep pushing i / I keep changing i unil an insigh drops ou. In he case of he genre exploraion, he ool being developed helped me sumble on somehing only angenially relaed o my direc scienific work. This happens all he ime in oher scienific disciplines. Many, hough, consider sofware and programming a hard-headed ool like a shovel and herefore no suied for direc exploraion. This is silly, of course. Curren mehodologies avoid my ype of programming. I isn sofware for hire where business value is being creaed, nor is i defense sofware. Defense consideraions he code mus work led o he heavy mehodologies. Corpus Classficaion Corpus Classificaion Kno (P) Poery Gribble / Fedora (P) Poery Trakl (P) Lanier (P) The Waseland (P) Moby Dick (F) Gay Sories (F) To Kill a Mockingbird (F) Hamle (?) Berrand Russell (NF) Charles Babbage (NF) Darwin (NF) Crazy CS Person (NF) Bible (?) Pee Turchi s New Book (NF) Poery Poery Poery Ficion Ficion Ficion Ficion[Poery] Nonficion Nonficion Nonficion Jane Holmes / Humanophone (P) Jane Holmes / F2F (P) Fron Page NYT Aricle (NF) Richard Schmi / Kodiak (F) Richard Schmi / A Year of Counseling (F) Harper / Prac. Found. for Prog. Lang (NF) Ellen Bryan Voig / Song and Sory (P) Tennyson / In Memoriam (P) US Consiuion (NF) Tom Lux / I Love You Sweahear (P) Ficion[Poery] Poery Ficion[Nonficion] Poery[Ficion] Poery[Ficion] Nonficion Poery Poery Nonficion Ficion Poery rpg / Sharp Tone (P) Poery Ficion[Poery] Ficion[Nonficion] Cass Pursell / Men and Sones (F) Prous s Longes Senence (F) Ficion Ficion Figure 5 One of he lile secres abou business is ha firms are very conservaive. They wan o win by being beer, bu only enough beer o win no a lo beer. Business value is mosly abou caching up quickly. Rarely, I hink, is i abou being quick as a firs mover. Firs movers come from he kind of programming & science I am alking abou, and usually ha akes place in research labs, universiies where sarups are incubaed, ec. The rules I use make no sense for defense nor for business. Anoher purpose of InkWell is o serve as he sofware half of a wriing cenaur, a cenaur being a human/machine collaboraion. InkWell akes ex as inpu and a (largish) se of consrains, and produces a number of possible revisions while endeavoring o saisfy and balance he saed consrains. A wrier uses InkWell o assis wih revisions, and he workflow is ieraive wih he wrier creaing / weaking exual emplaes and consrains, and InkWell producing revisions, which feed back ino he process. The erm cenaur originaed in compuer chess, and refers o he pairing of a human chess player and a chess-playing compuer, usually a PC or lapop. Garry Kasparov [17] came up wih he idea of such collaboraions, and he chess communiy supplied he colorful and meaphorical name. There are some major differences beween a chess program and InkWell as compuer halves of cenaurs. The chess playing 8
compuer helps avoid blunders he human migh make. Ink- Well suggess avenues of exploraion he human migh miss. There are wo goals InkWell serves: mimic a specific wrier assis creaiviy in wriing InkWell akes a emplae (example in he Appendix ([Figure A1]), which is a specificaion of original ex annoaed wih which words are variable and characerisics of hose words for InkWell o consider. There are also a number of oher, wriing-relaed consrains wrien eiher as local bindings in he emplae or saed in he UI, which specifies global parameers and consrains. The example in [Figure A1] shows how a wrier migh express a emplae describing Rober Fros s Sopping by Woods on a Snowy Evening [15]. Here are some of he general ways o mimic a wrier: mach specified (or measured) Big Five personaliy rais and associaed personaliy faces; mach basic human values as described by Schwarz [18] and Chen [19] mach a wrier s word choice: favored words, word music, word lengh, favored mood mach wriing paerns: n-grams (2-, 3-, 4-, and 5-grams); an n-gram is a series of n words in a row ha has appeared in a naurally occurring, exising ex Assis Creaiviy: every wrier has days when hey have i and days when hey don. Books, aricles, blog poss, courses, coaches, and workshops exis o help wriers defea wrier s block. Or some have jus a lile less alen han preferred. InkWell can help here oo, using hese echniques: use conservaive or wild synonym choice (associaive versus dissociaive wriing) search diameer, search disance, preference for nearby, preference for far away, various synonym aspecs (hypernyms, meronyms, ec) saisfy consrains like word-lengh, alernaive meanings, word rhyhms favor echoes (similar sounding words) and rhymes selec words based on onology (conceps), proximiy in he synonym nework, or a cluser of word-cenric conceps o favor or avoid favor specific word groups or avoid hem specify consrains, boh local and global ake ino accoun a wriing mood specified by a consruc called a halo Any consrain can be invered: e.g. sound like a paricular wrier or sound like anyone bu ha wrier, rhyme wo words or ensure hey don, observe n-grams or deliberaely violae hem. InkWell produces any number of candidae revisions, and he wrier can pick and choose revisions and wordings. The noion of a halo is a good example of mimicking wrierly hinking. A halo is a mood device. You specify a se of words, and InkWell sars wih each of hose words and fans ou along synonym arcs o oher words. Where several of hese wavefrons hi, hose words are given more weigh in he revision process. Looking a Fros s poem, he line The woods are lovely, dark, and deep is revised his way The woods are brigh, ligh, and high when given he happiness halo: Delighed, Ebullien, Ecsaic, Elaed, Energeic, Enhusiasic, Euphoric, Excied, Exhilaraed, Overjoyed, Thrilled, Tickled pink, Turned on, Vibran, Zippy and his way The woods are ho, rough, and cold when given he anger halo: Affroned, Belligeren, Bier, Burned up, Enraged, Fuming, Furious, Heaed, Incensed, Infuriaed, Inense, Ouraged, Provoked, Seehing, Sorming, Truculen, Vengeful, Vindicive, Wild A long ime ago (~1980) I wroe a simple NLG sysem my PhD hesis was his: a generalized planning sysem based on loose descripions of individual agens, heurisic maching, resource-limied compuaion, and mixed planning and execuion could do a good job of producing ex [20]. The sysem was called Yh, and i was a small-daa program. I was abou 75,000 lines of code and had maybe 10,000 dicionary enries and language-relaed agen descripions. Yh was used as he ail-end of an auomaic programming sysem (called PSI [21]) a he Sanford Arificial Inelligence Lab o describe in English he programs produced (how hey worked) and in he mixed-iniiaive user dialog ha gahered he specificaions for he programs o be generaed. InkWell is differen; i has hese pars: WordNe synonym dicionary: 160,000 words [22] [23] 5,000 mos common words CMU phoneic dicionary : 125,000 words [24] rhyming dicionary: 42,000 words sem dicionary: 163,000 enries (+ Porer Semmer + Lemmaizaion) 9
n-grams: 30m from general lieraure; 100,000 1,000,000 per wrier including he Google 2-grams [25] and he COCA 3-, 4-, and 5-grams [26] ~30,000 lines of code: emplae compiler, consrain opimizer, word & phrase adjusmens, ec n-grams (including 1-grams) from a specified wrier; currenly here are around 50 wriers o choose from (and supplying new ones is rivial) emplae wih consrains compiled evaluaion funcion A naïve flow diagram is o he righ [Figure 6]. A emplae resembles a Lisp program whose body looks like ex wih parenhesized annoaions. The emplae along wih all he specified consrains and parameers is compiled ino an evaluaion funcion which reurns 0 when all consrains are saisfied. InkWell selecs a se of candidae replacemen words and phrases, and an opimizaion process hen selecs he combinaion of words and phases ha bes saisfy (minimize) he evaluaion funcion. The opimizaion process uses simulaed annealing mosly because he opimizaion is over discree word choices and I haven been able o find a beer process. (One nice characerisic of simulaed annealing is ha I don have o worry abou saging he order of choices I can jus add consrains and le he relaively undisciplined SA process do is hing.) This produces (as many) revisions (as he wrier wans); he wrier can pick and choose which suggesions o use, and he emplae can be alered o do more focused exploraions. For example, o evaluae how close a se of word choices is o a specified personaliy profile, InkWell analyzes he proposed ex o (compuaionally) deermine is exhibied personaliy. This is hen compared o he arge, and eiher he curren ex is considered a sep in he righ direcion or a sep in he wrong direcion. Because he analyzed exs are generally prey shor, he LIWC dicionary needed o be expanded, which was done algorihmically using he WordNe synonym dicionary, a decay funcion, and semanic caegories derived from he LIWC caegories his expanded he LIWC dicionary from abou 2,300 words o abou 25,000 words his is now he dicionary my group uses. The compuaion of personaliy scores needs o be fas, which is accomplished by pre-compuing as much of he LIWC caegories as possible and using caches. Le s look a some oupu. I ll use he familiar Fros poem [15], bu we ll look a only he las wo sanzas. The Lisp [27] UI (and yes, I know he UI is ugly and horrible) for InkWell is shown in he Appendix. So is he original poem. Here are he original las wo sanzas: 10 Figure 6 He gives his harness bells a shake To ask if here is some misake. The only oher sound is he sweep Of easy wind and downy flake. The woods are lovely, dark, and deep. Bu I have promises o keep And miles o go before I sleep, And miles o go before I sleep. The firs revision specifies conservaive synonym search (don raverse far from he originally specified words), and InkWell is insruced o prefer word choices ha Ernes Hemingway [28] used in his shor sories as well as shor words. Here is ha resul (changes from original underlined): He gives his harness bells a shake To ask if here is some misake. The only oher sound is he brush Of comforable wind and small snowflake. The fores is lovely, off, and abundan. Bu I have promises o keep And miles o go before I sleep, And miles o go before I sleep. The second varian specifies wild synonym search (go far and wide), and Wal Whiman [29] is he wrier o sound like (& use shor words): He gives his work values a run To say if here is some misconcepion. The only oher being is he brush Of simple appearance and new person. The lands are divine, cerain, and low. Bu I have loves o keep And runs o go before I go, And runs o go before I undergo. opimizaion revisions
The las example of hese sanzas is my favorie. I s like he Whiman one excep I ve asked InkWell o sound a lile like a very heoreical and a bi conroversial compuer scienis [30]: He gives his direcion presenaions a can To hold if here is some misconcepion. The only oher elemen is he exi Of informal erminaion and yielding person. The developmens are saisfying, insensiive, and exended. Bu I have complexiies o erminae And amouns o go before I separae, And amouns o go before I admi. In addiion, InkWell was insruced o prefer long words. Looking a he emplae in he Appendix, you can see some ineresing hings hings ha are specified as sof consrains: InkWell ries o mach he currenly loaded wrier s Big Five personaliy rais; for he las example his is he compuer scienis; call his he arge wrier he arge wrier s word choices and n-grams are mached InkWell is old o make as many chosen words echo as possible; wo words echo when he share word sounds his is derived from he CMU Phoneic dicionary InkWell ries o make all he words i chooses differen from each oher for he examples ha arge Hemingway and Whiman, InkWell is asked o prefer shor words; his is in erms of syllable coun, as derived from he CMU Phoneic dicionary he bind saemens are like Lisp s le; hey bind variables o a word-choice specificaions, and all occurrences of he same variable in he body of he ex will be replaced by he same word; e.g., w2 always refers o he same word or phrase ha means snow variable words are called ou; hey are annoaed wih heir pars of speech (including semanic caegory, if known (e.g. verb-cogniion)) and oher consrains many of he consrains are easily undersood; for example his expression (ref w2 :differen w1 :rhyme w1) means he word seleced o mean snow should be differen from he word seleced o mean know, and ha hose wo words should rhyme; he expression (ref w3 :echo w1) means ha he word seleced o mean queer, odd, or unusual should echo he word seleced o mean snow ; he expression (mile noun-quaniy pl) means he word chosen o mean mile should be a noun in he semanic caegory quaniy, and ha when ha word is expressed in he final oupu, i should be made plural ohers are no so clear; he expression (sleep verb :differen sleep :rhyme sleep) means ha he word seleced o mean sleep here shouldn be he same as he word seleced in response o he binding for sleep (he las word of he previous line), bu should rhyme wih i; he expression (wood 11 noun-plan pl :+sense [fores] :-sense [wood]) means ha he word chosen o mean wood should be in he semanic caegory plan, should be made plural, should be of he same sense as he word fores and no he same sense as wood, (he maerial rees are made of) his is achieved by saring a he word fores and spreading ou, increasing he srengh of each word encounered by an amoun ha decays oward 0 wih disance (he amoun sars posiive for :+sense words and negaive for :-sense words) predicaes can be specified for each word, and a pervasive predicae is one ha applies o all of hem; he predicae syllable-bonus-few indicaes a preference for shor words The n-grams are used o ry o mainain some degree of familiariy and sense. Wha did I sar wih if I didn sar wih hard requiremens or a spec? I had had a en-year hobby of using simulaed annealing o solve puzzles and do conference room scheduling. I had developed several SA frameworks, and i was fun and producive. As a wrier I had a sense ha I balanced a lo of concerns while wriing, especially poery. I fel ha hese concerns were no srucured and ha I didn consider hem in a paricular order. I sared wih he idea ha I could represen a paragraph as a sequence of boxes conaining words, phrases, and oher hings, ha each box could have an associaed se of alernaives, ha I could represen he concerns as numerically measurable feaures viewed as sof consrains, and ha SA would be able o do is magic o balance all hose concerns o selec he bes alernaives. My firs consrains were he Big Five personaliy measuremens, he n-grams, and he proximiy of synonyms o original words. All he oher consrains in InkWell fell ou of rying o hink of how o measure numerically he srengh of a (possible) craf elemen how much do hese wo words rhyme, does his word beer mach a given semanic sense, is his word more like wha a paricular wrier would use. Experience wih InkWell guided me, and I d say i led me by he nose. Kasparov lamens our urning away from mysery and focusing on puzzles. He doesn pu i exacly ha way: This is our las chess meaphor, hen a meaphor for how we have discarded innovaion and creaiviy in exchange for a seady supply of markeable producs. The dreams of creaing an arificial inelligence ha would engage in an ancien game symbolic of human hough have been abandoned. Insead, every year we have new chess programs, and new versions of old ones, ha are all based on he same basic programming conceps for
picking a move by searching hrough millions of possibiliies ha were developed in he 1960s and 1970s. Garry Kasparov [17] Does his also say ha in urning away from creaiviy / mysery we have urned o puzzles? Turned o providing immediae value o firms? Kasparov goes on: Like so much else in our echnology-rich and innovaion-poor modern world, chess compuing has fallen prey o incremenalism and he demands of he marke. Brue-force programs play he bes chess, so why boher wih anyhing else? Why wase ime and money experimening wih new and innovaive ideas when we already know wha works? Such hinking should horrify anyone worhy of he name of scienis, bu i seems, ragically, o be he norm. Our bes minds have gone ino financial engineering insead of real engineering, wih caasrophic resuls for boh secors. Garry Kasparov [17] Whenever I add a new consrain ype o InkWell I have o go hrough a period of geing re-acquained wih i. InkWell operaes in a very complicaed space of consrains, and finding he good spos akes ime. Afer revising InkWell, I mus revisi all my personal heurisics abou how o wrie emplaes o ge i o be creaive, cauious, wild, or conservaive. I s a pleasurable re-familiarizaion kind of like geing o know a new lover: hings roughly work he same, bu all he deails and nuances are fresh and exciing. I ypically allocae a few days o do his, and I ofen send off some of he resuls o my poe friends for heir amusemen. InkWell is in Common Lisp. I wrie in Common Lisp because I know i well, know how o do crazy hings wih i, and i flows from my fingers rapidly. InkWell is complex: maybe half a dozen compilers (wo very subsanial), he opimizaion infrasrucure and InkWell manifesaion of i, he synonym machinery, he consrain compuaion machinery, he parallelizaion o make he synonym discovery and opimizaion run accepably fas, he numerous caches o make hings run fas enough o be olerable (even wih he parallel suff). For example, he emplae shown in he Appendix for he Fros poem compiles o a Lisp funcion 1952 lines long. This funcion is composed of he following: 16 (saic) calls o he rhyming predicae a (saic) call o a funcion ha measures how echo-y a se of 44 words are a (saic) call o a funcion ha measures how diverse he same se of 44 words are 126 (saic) calls o a funcion ha deermines wheher pairs of words are known 2-grams 12 120 (saic) calls o a funcion ha deermines wheher riples of words are known 3-grams 5 (saic) calls o he LIWC / Big Five personaliy compuaion This funcion is run in he inner loop of he opimizaion process. The emplae specificaion in he Appendix runs i a million imes. Tha means ha he funcion ha deermines wheher a pair of words in he revised ex is in he Google 2-grams se is called 126,000,000 imes. An example of he way he evaluaion funcion works is rhyming. Suppose a emplae specifies ha he words seleced for he variable words v1 and v2 should rhyme. When wo acual words, w1 and w2, are chosen, hey are passed o a funcion (120 lines of Lisp code including subfuncions) ha compues a rhyming score, 0 r 1, for hem, based on an algorihm for rhyming. Roughly, ha algorihm looks a all phoneic spellings of w1 and w2 in pairs, sars a he ends of each pair, and compues how much each syllable rhymes along wih how many syllables rhyme. The maximum rhyme score for all pairs is chosen for r and hen flipped (1-r) so ha a perfec rhyme yields 0. All he sof consrains are reaed his way. All such evaluaions are summed, and he overall evaluaion funcion is minimized over all word choices. This approach enables InkWell o selec hese hree delighfully unexpeced words as rhymes a he end of he Fros poem: gulp, hole up, and nap. To make his run fas enough o be usable, here are muliple layers of caches ha memoize hese funcion calls and pars of hem. I could probably figure ou how o make a more compac evaluaion funcion for my needs perhaps by finding oher ways o compue he consrains or heir equivalens. Bu he combinaion of sraighforward compuaion and los of caches makes experimens easy even if he code is complicaed. Given his, I was able o do some ineresing invesigaions ha migh no have been possible were hings more sreamlined. For example, afer I sumbled across he CMU Phoneic Dicionary [24] and came up wih he rhyme-srengh algorihm, I was able o approach he poeic concep called echoes. Before his discovery I had no realisic approach for how o measure i. Laer I will be able o analyze rhyhm because he phoneic dicionary scores sress levels for syllables. The words seleced need o be fleshed ou for oupu. For example, recall ha he word for mile needs o be urned ino a plural. Moreover, he WordNe synonym dicionary conains phrases as well as words. Handling he deails for all hese adjusmens akes a pile of algorihms, many ables, and complicaed special cases. The code for his par of he sysem is 1000 lines righ now, and I generally add o i / revise i wih every oher new emplae I ry.
My mehods of exploring how o ge a program o choose words and phrases varied over ime depending wha my collaboraor, InkWell, was eaching me. I always would sick wih he basics of he scienific mehod, bu I would chase hunches, build unlikely infrasrucures o explore wha seemed like dead ends, and generally would use he heurisic of looking inenly, from ime o ime, a he leas likely idea. InkWell iself as well as my old NLG program Yh does he same hing wih respec o abandoning common sense. In InkWell, simulaed annealing works (well) because i will occasionally / randomishly choose o make changes o he sae of affairs ha make hings worse. This is likely he reason i s so good a finding unusual rhymes. In Yh I used a echnique I called counerinducion afer he philosophical concep of he same name [31]. The idea was ha when planning progress is advancing slowly or no a all, Yh would allocae a lo of resources o explore less likely approaches. Kind of like a chess program ha can do heurisic esimaes of poenial nex moves choosing o explore o some deph he consequences of making a move he heurisics don like. Here is how he biologis Kim Lewis pus i: This is par of wha I each my sudens how o shu down your common sense. You have o sar looking for a perfec soluion and ignore wheher i s realisic. Tha mindse helps you bale your common sense, which is wha prevens you from invening new hings. Kim Lewis[32] Gabriel, you loser we have a word for his in agile i s called a spike. Someimes a user sory is generaed ha canno be esimaed unil he developmen eam does some acual work o resolve a echnical quesion or a design problem. The soluion is o creae a spike, which is a sory whose purpose is o provide he answer or soluion. Like any oher sory or ask, he spike is hen given an esimae and included in he sprin backlog. hp://www.soluionsiq.com/resources/glossary/bid/56550/ Spike [33] I s funny how when I alked abou his wih audiences in he pas and go he you loser commen, I couldn really answer he criicism. The answer is in he above quoe bu only oday while wriing his ex did I realize i. Resolve a echnical quesion or a design problem. A spike soluion, or spike, is a echnical invesigaion. I s a small experimen o research he answer o a problem. For example, a programmer migh no know wheher Java hrows an excepion on arihmeic overflow. A quick en-minue spike will answer he quesion. Shore & Warden, The Ar of Agile Developmen [34] Agile is aimed a assising a business guy creae business value righ away. The developers are no ineresed in figuring ou wha ha business value is, bu simply wish o hear i old o hem. A ha 2001 meeing in Snowbird where we wroe he Agile Manifeso, Ken Beck saed one of our goals: o heal he divide beween developmen and business. Rober Marin, The True Corrupion of Agile [35] My goal is o resolve a mysery, bu i s no a echnical quesion abou how o design or code InkWell hough I have pleny of hose i s a mysery abou wha makes for creaive and arisic naural language generaion. Spikes are deours developers ake o figure ou hings abou he programs hey are wriing; hey are no deours o figure ou wha business values o pursue. Confusing spikes and Naur s heory building for wha I am up o is he same misake wice. Alrigh friends, you have seen he heavy groups, now you will see morning maniac music. Believe me, yeah. I s a new dawn. Good morning, people! Grace Slick, Jefferson Airplane, Woodsock, Augus 16, 1969. We arrive a he crux. Wha is programming? I s easy o be confused by hings like his for example: A sofware developmen mehodology or sysem developmen mehodology in sofware engineering is a framework ha is used o srucure, plan, and conrol he process of developing an informaion sysem. hp://en.wikipedia.org/wiki/sofware_developmen_ mehodology [36] Sofware developmen involves programming, bu sofware developmen isn programming. Mehodologies are abou appropriae ways o develop sofware in an engineering-relaed conex. For miliary purposes and for safey-criical purposes, i s essenial o no make a misake, and he way o do ha is los of concepual planning near he beginning o be cerain nohing can go wrong. This leads o he heavy mehodologies. In business conexs he high-order bi is o ge working sofware fas, and here is a premium for helping firms cach up quickly o compeiors, veer ahead wih a new (bu ypically incremenal) produc, or respond o sudden changes of direcion he cusomer migh hrow ino he hopper. Bu as 13
wih he miliary conexs, he endgame is a sofware-relaed arifac in he domain s real world doing hings ha someone needs or wans. There are oher conexs. One is science bu a paricular ype of science. I s no science where daa gahered from insrumens is analyzed. I s when he sofware and is programming form a machine o explore naure alongside he scienis. I could be a simulaion ha helps he scienis undersand wha s going on, or i could be like InkWell which is rying o creae an arificial model of wha wrierly creaiviy is. Anoher, similar, conex is learning abou a opic hrough programming i up. For example, my abiliy o undersand los of echnical hings is limied, and so I program hem up myself o see how he mechanisms lock ogeher o make i happen. Anoher is o explore wha a quesion ha s easy o ask migh acually mean. For example, as par of he InkWell projec I am exploring wha rhyminess could mean. In wrier circles a paricular wrier migh be considered more (or less) musical han anoher; wha does ha mean? My firs answer was ha i is he percenage of words in a ex ha rhyme. I programmed ha up and found he idea doesn ake ino accoun a ex ha is exremely rhymey in a couple of isolaed places and oherwise fla. My nex idea was o ake a window abou 100 words wide and pass ha over a wrier s corpus skipping ahead 50 words a a ime (ha is, overlapping he windows), compuing he percenage of words ha rhyme in each 100-word group, and reporing he average of hose percenages. This resuled in scieniss and nonficion wriers being very rhymey because here would be knos of high-rhyme bundles, usually because los of echnical words rhyme for no aseful reason. I noiced ha people considered rhymey had narrower sandard deviaions han fla wriers for he se of rhyminess windows. So I ried a formula where a wrier s rhyminess is he average rhyminess minus wice he sandard deviaion. The resuls are prey inuiive, bu I hink here is a sronger noion of periodiciy a work ha needs o be considered. (There are some rhyminess scores in he Appendix.) The reason I old his rhyminess sory was o demonsrae ha he quesions being explored wih sofware his way are no sricly abou analyzing daa (like creaing a simple word usage model of a wrier) nor how o achieve he requiremens of he program (how o implemen a heory of rhyminess), bu wha a concep migh mean. I migh seem ha he programming sories I old are jus a se of projecs I worked on exending he LIWC dicionary, programming up a lierary genre deecor, algorihmic rhyming, exploring he concep of rhyminess, and ailoring exs using opimizaion bu hey are jus waypoins on a single journey o discover how naural language generaion can be done and wha creaiviy means in wriing. InkWell and all is inner suff is he machine I lug around wih me as I explore his space. I s my learning machine. I program o explore. If you learned somehing from his essay I would be very disappoined. I ve simply poined ou ha programming is no sofware engineering, and because of ha, he principles and pracices of he heavy mehodologies and agile are oo limiing and even irrelevan. Programming is like wriing in ha sense. You can wrie a 5-paragraph heme for homework, you can wrie a requiremens documen for your division, you can wrie a markeing piece for your produc manager, you can ake effecive wriing courses. Or you can do his: I wrie enirely o find ou wha I m hinking, wha I m looking a, wha I see and wha i means. Wha I wan and wha I fear. Joan Didion, Why I Wrie [37] Or his: You may wonder where plo is in all his. The answer is nowhere. I believe ploing and he sponaneiy of real creaion aren compaible. I wan you o undersand ha my basic belief abou he making of sories is ha hey prey much make hemselves. The job of he wrier is o give hem a place o grow. Sephen King, On Wriing [38] Or his Bu during my very early wriing, cerainly before I d published, I began o learn characers will come alive if you back he fuck off. I was exciing, and even a lile errifying. If you allow hem o do wha hey re going o do, hink and feel wha hey re going o hink and feel, hings sar o happen on heir own. I s a beauiful and exciing alchemy. And all hese years laer, ha s he hrill I wrie o ge: o feel hings sar o happen on heir own. So I ve learned over he years o free-fall ino wha s happening. Wha happens hen is, you sar wriing somehing you don even really wan o wrie abou. Things sar o happen under your pencil ha you don wan o happen, or don undersand. Bu ha s when he work sars o have a beaing hear. Andre Dubus III, By Hear [39] Tha s why I wroe his essay o find ou. 14
Acknowledgmens Some of his work was suppored by DARPA (W911NF-12-C-0028). Appendix: Code Examples & Suff (wih-personaliy-rais (*wrier-big-five*) (wih-global-consrains ((all-echo)(all-differen)) (wih-pervasive-predicaes (# syllable-bonus-few) (bind ((w1 (know verb-cogniive)) (w2 (snow noun-subsance)) (w3 (or (queer adj) (odd adj) (unusual adj))) (w4 (or (year noun-quaniy) (week noun-quaniy) (monh noun-quaniy) (season noun-quaniy))) (w5 (shake verb)) (w6 (flake noun)) (here (here adj)) (near (near adj)) (mile (mile noun-quaniy pl)) (sleep (sleep verb)) (woods (wood noun-plan pl :+sense [fores] :-sense [wood]))) Whose (ref woods) hese are I (hink verb-cogniion) I (ref w1). His (house noun) is in he (village noun) hough; He will no see me (sop verb gerund) (ref here :rhyme near) To (wach verb-percepion) his (ref woods) (fill verb) up wih (ref w2 :differen w1 :rhyme w1). My (lile adj) (horse noun-animal) mus (hink verb-cogniion) i (ref w3 :echo w1) To (sop verb) wihou a (farmhouse noun) (binding near :rhyme w3) Beween he (ref woods) and (frozen adj) (lake noun) The (darkes adj) (evening noun) of he (ref w4 :differen w3 :rhyme w3). He gives his (harness noun) (bell noun pl) a (ref w5 :echo w3) To (ask verb) if here is some (misake noun :rhyme w5). The only oher (sound noun) is he (sweep verb) Of (easy adj) (wind noun) and (downy adj) (ref w6 :differen w5 :rhyme w5). The Bu And And (ref woods) are (lovely adj), (dark adj), and (deep adj). I have (promise noun pl) o (keep verb :rhyme sleep) (ref mile) o go before I (ref sleep), (ref mile) o go before I (sleep verb :differen sleep :rhyme sleep). )))) Sopping by Woods on a Snowy Evening Whose woods hese are I hink I know. His house is in he village hough; He will no see me sopping here To wach his woods fill up wih snow. He gives his harness bells a shake To ask if here is some misake. The only oher sound s he sweep Of easy wind and downy flake. My lile horse mus hink i queer To sop wihou a farmhouse near Beween he woods and frozen lake The darkes evening of he year. The woods are lovely, dark and deep. Bu I have promises o keep, And miles o go before I sleep, And miles o go before I sleep. rpg 15
Figure A1 16