Modelling and Measuring Collaborative Software Engineering

CarlCook

NevilleChurcher

SoftwareEngineering&VisualistionGroup,DepartmentofComputerScience&SoftwareEngineering,

UniversityofCanterbury,PrivateBag4800,Christchurch,NewZealand

E-mail:{carl,neville}@cosc.canterbury.ac.nz

Abstract

CollaborativeSoftwareEngineering(CSE)supportsthefine-grainedreal-timedevelopmentofsoftwarebyteamsofdeveloperslocatedanywhereontheInter-net.InthispaperwedescribeCaise,ourCSEenvi-ronment,andexplorethewaysinwhichsuchenviron-mentscanbenefitdevelopers.Weconsidertherˆolesofheuristicevaluation,loganalysisandvisualisationinquantifyingthebenefitsofCSE.

Keywords:CollaborativeSoftwareEngineering,CSCW,Evaluation1

Introduction

Modernsoftwareengineeringinevitablyinvolvesteamsofdeveloperscollaborativelyworkingonsoft-wareartifacts.Challengestosuccessincludeboththesize(millionsoflinesofcode,thousandsofclasses)andcomplexityofthesoftwareunderdevelopment.However,increasingly,individualdevelopersmaybephysicallyseparated—perhapstotheextentthattheyareindifferenttimezones.Similarly,thesoftwareartifactsmayalsobearbitrarilydis-tributed/replicatedatthedeveloper’slocations.Sup-portingeffectivecollaborationbetweensoftwareengi-neersisitselfadifficultproblem:supportingeffectivecollaborationbetweenphysicallyseparatedsoftwareengineersremainsanopenproblem.

TheseproblemsareaddressedinthefieldofCol-laborativeSoftwareEngineering(CSE),whichwede-finelooselyasseamless,fine-grained,real-timecollab-orationbetweendistributedsoftwareengineerswhomaybelocatedanywhereontheInternet.Inprac-tice,thisinvolvesthedevelopmentoftools,techniquesandenvironmentswhichminimisetheadverseeffectsofcollaborationwithremotecolleagues.

Softwareengineeringinvolvesteamworkandcom-municationofmanykinds.Specificexamplesinclude:•Inagileprocesses,suchaseXtremeProgram-ming,pairprogrammingrequiresveryclosecol-laborationfocussedonthesameartifact.InCSE,thepairmembersneednotbespatiallyco-located.

•Developmentactivitiessuchasanalysis,design,testingandcodingmaybecarriedoutbydiffer-entcombinationsofindividuals.CSE-mediateddiscussionsarepotentiallyavaluablewayforef-fectivecommunicationandfeedbackbetweenandwithinthesegroups.

timescales.CSEThiscouldbesupportedinrealtimemodellingInenvironment.

inathispaperweorderandtoprovideCSEactivities,considertheissuesassociatedwithempirically-groundedassessingCSEsystemsinillustratetheanalysisourCaiseoursystem.

arguments&visualisationwithexamplesofCSEdevelopmentactivity.drawnfromWelows.ThebetweenInremainderthenextofthispaperisstructuredasfol-temitanalysissupportsispresentedCSEandsection,CSCWwesystems.discusstheOurrelationshipsCaisesys-CSEinareSectionoutlined.3andSectiontheways4containsinwhichanlaborationsofthedevelopwhichmajorareactivities,evidentinrelationshipsCSE.InSectionandcol-bilitiesbeandaseteffectivenessofheuristicsofwhich5weCSEwillenablethecapa-iesassessed.longitudinalofbothCSEWeshowinSection6implementationsthateffectivestud-tologplesdata.WestudiesandwhichCSEneedsystemstoberequiresupporteddetailedbyconclusionsfromCaiseillustrateourargumentswithexam-researchistaking.andapplications.indicatethedirectionsFinally,weourpresentongoingour2

CSEandCSCW

Onetributedapproachtotheproblemofmentsoftwareengineeringhasbeensupportingthedis-&ofconfigurationmanagementtools(Conradidevelop-portedWestfechtel1998)andtheseareroutinelysup-Programmingbymodernware(XP)IDEs.TheadventofeXtremetools.development(Beck2001)andotheragilesoft-tionCVSinprocessesreliesheavilyonsuchprojectsofweb-basedparticularhasenabledtheprolifera-alone(SourceForge(over86,000open2004,aresourceBarregisteredsoftwaredevelopment&FogelatSourceForgevaluable,However,toolsalthoughsuchthesetheycanhaveonlyproved2003)).

partiallyextremelysupportCSE.ordinationTypically,thepricepaidbydevelopersforco-modelsoftheireffortsisacceptanceofextremeaccessApessimisticofconcurrentmodelactivity.

usesment.tothoseTheartifacts:underlyingthisminimiseslockingtopreventsharedassumptionsparallelareeffectivelydevelop-usersofatypicaldatabaseapplication—thatmultipleartifact(transactions)flicts.

arelikelytoleadconcurrentlytointegrity-threateningaccessingthesamecon-performBycontrast,Itindependentanoptimisticupdatesmodelallowsuserstopotentialconcentratesmaximisesconflictsonwhenprovidingtothesameartifact.updatessupportareforresolvingchangesparalleldevelopmentbutrisksmerged:conflictingthisfoundBothwhichpessimisticcanbedifficulttoresolve.

Clearly,inthustoolstypicalsuchconfigurationandoptimisticapproachesareasCVShavemanagementtools.istiveideallyfar.However,suitedtowefine-grainedcontendthatbeenverysuccessfulreal-timeneitherextremeplicitlysoftwareengineering.Pessimistictechniquescollabora-ex-(i.e.developmentunawareforceofuserseach(transactions)other)—farfromtobeidealindependentinateamtoperformed.

unpleasantcontext;surprisesoptimisticwhenintegrationtechniquesiseventuallycanleadIndividualNeithermethod,changes,extremeissuchparticularlyassuitableforCSE.project.tifactsIfmayuserstouchaddingaparametertoahavesignificantfractionsofthewholeinrequired(whichtomaywaitnottonecessarilycheckoutallbetheknownar-otheradvance)hand,somethenproductivitychangesmayisbelowered.complexOnandthein-

volveapproachmodificationsconflictsrequiresthatinseveralthesebeplaces.backedIfanoptimistictermCSCW,thenalsoitmaynotbeeasytoreproduceoutbecausethem.oftoforsoftwareknownwhichasallowsgroupware,userstoiscollaborateagenericReinperformwhiteboards1991,fortasksorexample).ordeveloptexteditorsTypicalartifactswhichexamples(Ellis,Gibbs&allowusersaresharedatdif-ferentoftheInternetlocationsnodesandactionstomakeofchangesotherusers.

andtobeawareitsAsofarewellavailableasspecifictosupportapplications,anumberoftoolk-Groupkit,groupwareGroupLabdevelopedapplications.thedevelopmentofsuitesattheOnenotableexampleistrolCentral(Rosemantothisis&GreenbergUniversityofCalgary’stheconcept1996).

&interleavedDewan(Dommel1996).&Garcia-Luna-Acevesoffloorcon-Floorcontrolpolicies1997,Munsonmaysuchbeasrestrictiveactionsofasconcurrentusers.mediateWhilethesetherare.aswhereLess2-phaseapessimisticlockingschemerestrictivelocking,suchpoliciesarerelativelybeforeuserstheycanmustobtainpoliciessomeincludetokenorturn-taking,permissionfree-for-all,However,flictingpermittingthemakemostupdatesusualpoliciestosharedareessentiallyartifacts.athatwillsocialupdatesprotocolsatwill.userswillTheensureunderlyingtomakepotentiallythatgroupassumptioncon-membersisharmfulconsultuserschanges.oneanotherbeforemakingpotentiallyVOIP),caninteractviaInaaudiotypical(telephoneCSCWenvironment,conferenceorthevideoandtextchatfacilitiesaswellisIngroupwareasviaorderforapplications.

tions,necessaryforsuchusersapproachestobemadetoawarebesuccessfulitexample,actionssuchprovidesandintentionsoftheloca-avarietyofothers.Groupkit,forscrollasmultiplecursors(telepointers)ofawarenessandmechanismsmulti-userengineeringThebars.

applicabilitycontextsfor(ChurcherhasbeenofCSCWtechniquestosoftware&demonstratedforsomespecificindependentexample).approachCSCWEarlyattemptsCerecke1996,Schummer2001,components.typicallyuseddomain-temswell(GreenbergwassufficienttocopewithWhilesmall,simplethisna¨sys-ıveGrudintolarger,more1989),complexthetechnologysituations(Grudindidnot1992,scaleinsuch1994)andCSCWhaslargelybeenceptsHowever,contexts.

abandonedourapproach,whichmuchontoacoreIDEengine,isrobustlayersandCSCWpotentiallycon-singlemorecopyofappropriateallartifacts,formanagesCSE.Caisesyntacticmaintainsandse-amanticandareperformsmodels,avarietydetectsofandpropagateseventstouserscomponentssupportedshipsoftheunderlyingtheybyaareserverothermanipulatingthatfunctions.fullyunderstandsClienttoolsthesoftwaremodel.

andtherelation-UMLToolUMLToolTextTool...TextTool...CarlInternetNevilleCVSLoggingToolserverCAISE Server...Figure1:Collaborativesoftwareengineering

Client ToolsInputEventsParsersParseTreeSemanticModelAnalysersModelChangeEventsFeedbackModulesFeedbackEventsint iint iOutput EventsEvent DistributorLocal Views of ModelXML Event LogCAISE ServerFigure2:EventpropagationintheCaisearchitecture

ourFigurecatedsystem.1developmentatdifferentIndividualillustratesInternetusers,thehighlevelarchitectureofnodesCarlandandeachNeville,arelo-mingtoolssuchascodeeditorsandhasdiagram-asetofservertools.return,ofupdatesThesetheyindividualtoolsinformtheCaisetifactsusersasarethetheyrefreshedserverensuresmakeoccur.toreflectthattochangestheirprojectlocalartifacts.Inmadecopiesbyofotherar-3

TheCaisearchitecture

Theclient/serverarchitectureofCaiseispresentedinschematicallyFigure2andsupportstheCSEactivitiesshowntities:andsemanticCaiseinmodel.

server,Figureclient1.Ittools,compriseseventfourpropagationcoreen-CaiseallserverTheserverisresponsiblesociateduserswithinthesoftwareprojectforandmanaging

theiras-theartifacts,functionssoftwareofanartifacts.IDE:maintainingItperformsamanyoftracksupportingofreferencesconstructingbetweenasymboltable,cachekeepingofever,somethemultiplelanguagesunitsandoftools.code,How-andbetweensignificantCaiseways:serveritisdiffersfromanIDEin‘live’asversionmultipleofusersandsharedtools,simultaneouslyallowsonlyoneknowledgeacorefunctionartifacts,withclientandtools.

sharessupportsitscollaborationfullsemanticCentralsoftwaretotheserveristhesemanticmodelstoredrespondingby.Whilsttheserver,eachtheartifactmodelisstoresmanagedtheandofcor-asprojectsemantics:informationsuchandthethesymbollistoftable,currentlyreferencesunresolvedbetweenreferences.symbolsLanguage-specificintoandarepresentationsparsersmoretransformsuitableforsourcecodeficationfurthercorrespondingbyaprocessing.clientUponsourcefileexchangemodi-semanticparsetool,treetheinorderservertogeneratesupdateitsaisingalsoviews.

themdeliveredmodelandtoupdatetoinformusers.Theparsetreetheirinterestedlocalmodelsclienttools,andprojectallow-Theprocessserverrespondingtheupdateduseslanguage-specificparseanalyserstounderstandsemanticandboththemodeltrees,generatingcor-structureupdates.ofAnalysersofthesemanticmodel,andperformtheparsethetasktreebetweenupdatingtheoldtheandmodelnewtoparsereflecttrees.

thedifferenceUser-definedserveroruserfollowingfeedbackeventsthatchangesmodulesareinvokedbythesuggeststotheaclientsemantichaschanged

model,locationspectclient-specifictheincurrentanartifact.messagesstateoffortheFeedbackmodulesin-toolsmodeltoandrespondgenerateto.Allanusereventsuserartifact),change(suchevents,asachangefeedbackoflocationeventsinsages)communicationevents,arecollatedbyeventstheserver.(suchaschatmes-andtools,logbe.Thethewhereeventserverrelevant,logalsoaredistributedWhilsttoclienttheseisavailableappendsthemtoclientstotheeventbackusedforavarietyofpurposes,suchandasplay-canspecificofmetricsevents,theprojectminingdevelopment,forpatternsinspectionofusage,andofSection6.

gathering.SomeexamplesaregiveninInstoresadditiongetherdatatoaboutthesetallofprojectevents,userstheeventlogalsomentslog,siderableprovidingthewithindividualalldeclaredsymbols.Thisanddatatools,aug-to-theeventeventlogdataviewercontainedwithaintheeditingthesemanticofamountartifactsofmodel.

andinformationtheresultantrelatedchangestocon-thetoClienttoolstoolsCaiseclientsaresoftwareengineering

clientthateditor,toolconformartifactsthishaswillalocaltotheCaiseclientAPI.Eachtypicallymodel—inbethethesetcaseofofatextchangesserverandoccur,intheinturnthesoftwaresourcefiletoolsproject.WhenremoteupdatearetheirnotifiedownbymodelstheCaiseandcorrespondingdiagrammingtakestoolviews.couldForhaveexample,aaUMLclassparticularintoaccountupdatesetonlytheclasseslocalbelongingmodelthattoaspecificitsclasseslocalofoccur.modelpackages.whenThisremotetoolwouldupdatesonlytoEventthepropagationgetherreactionsTheactionsofclienttoolsand

openbytheofeventtheCaisemodel.serverWhenareboundto-eratedorthenandmodifydeliveredanartifact,inputeventstoolsarecreate,gen-ingrespondstotheevents,totheserver.typicallyThebyupdat-servereventsthefeedbackmaysemanticbegenerated.model—atFinally,whichpointuser-definedchangesponsealltoeventsspecificmaychangealsoandbegeneratedinre-toandtheeventsappropriatehavebeenlistenerscollated,inputsuchtheyevents.asarecodedeliveredOnceeditorstools.real-timevisualisation/projectmanagementSemanticmationmodelTherethoritativewithinareseveralsourcesofinfor-resentedstatetheCaisearchitecture.Theau-describedbythesemanticofthesoftwaremodel,whichprojecthasisrep-Thisincludeselsewherethecomplete(Irwinsyntactic&Churcherstructure2003)).beenof

theparseprojects,trees.

representedbythecurrentsetofParserationstreesholdtheexplicitstructureofalldecla-Additionally,andstatementselementifisstoredmodificationwithinasoftwareproject.intheparseinformationtree.foreachmethodaparsecreationdeclaration,treecontainsit(amongstispossibleotherForexample,toinspectthings)theaticdateofthemethod,plusanyseman-typechangeoraninformationadditionofasuchlocalasvariable.

achangedreturnAnmanticAPImightmodel.existsAtovisualisationinspectandtool,appendthese-tioninspectthemodeltogenerateforaexample,ilarly,oftheproject’scurrentclassstructure.visualisa-Sim-theport.

modelaprojectperiodicallymanagementtoproducetoolmightametricsinspectre-Ininstantiatedamorereactivegivenmanagementevents.withinmanner,Forexample,clienttools,eventlistenerscanbeareal-timewhichrespondprojecttoeventsandraisetoolamightwarninglistenwhentoamodelclassappearschangetoopers.

beinastateofconflictbetweenseveraldevel-Tologviewthefullhistoryoftheproject,theeventtractedmayfrombeinspected.theeventThelogdeclaredmaythensymbolsbelookedex-upvastintheandrelatedinformationsemanticcomponents.aboutmodeltheifsymbolrequired,declarationsproviding3.1

TheJavaeditortool

AnisexampleclienttoolwithintheCaisearchitecturecodepresentedinFigure3.ThistoolrepresentsaJavaeditoreditorconstructallowscurrentlyopenedbyuser‘Neville’.Thisorativelyandandanymaintainnumberinreal-time.aJava-basedofparticipatinguserstoThreemainprogramcomponentscollab-areidentifiedwithinthistool:

Figure3:ACaiseclienttoolinuse

Editorchronouspanel:groupware(labeledwidget‘A’inthatFigureallows3)multiple

asyn-distributedsamethetime.usersAsthetoCaiseeditagivenbufferatthelistenerspropagationcalcopiesinofreal-time,ofusertheartifactallinputarchitecturesupportseditorseventstointerestedbuffer.

maintainidenti-Usertooltree:(‘B’)providescontextawarenessforeditor.user,playWhilstandtheisparticularlyeditoronlyknowsusefulwithineach

thetheviewuserandtreesupportprovidesthemodificationofhowtexttofiles,dis-actualofsourcethesoftwareadynamicmodel-centricfilesbeingproject—independentedited.oftheClientthepanel:theartifacts(‘C’)withinprovidesthesoftwareinformationproject,relatedsuchto

asmodificationartifacteditorsvidesdates.andTheviewers,clientpanelfilesizesalsopro-andbacksendmessagesanareaforchatandvoicefromthethedisplaymessagesserver.ofcustomisedfeed-toItotherisalsousers.usedtoCaiseThethreeusedclientwidgetscomponentspackage;listedallabovesuchwidgetsarepartcanofthearchitectureMorewithindetailedanyJavaapplication.

beandassociatedinformationtoolsrelatedisavailabletotheCaiseelse-where(Cook&Churcher2003,Cooketal.2004).4

UnderstandingCSE

IneralorderandthetoassessqualitybothofthetheCaisebenefitsimplementationofCSEingen-inparticular,Caiseneedseveraltobeconsidered.

distinctaspectsofsystemssuchasArchitecture:chitecturecommonwhereasCaiseemploysreplicatedasystemsclient-serverar-AlthoughinciplessomeconventionalresearchhasCSCWbeendoneapplications.aremoreon¤tlyLindvallforevaluating2004,forarchitecturesexample),this(Tvedt,aspectCostaprin-isfactortheindeterminingaprimaryconcernourarchitectureforus.Thenotchoicemajorneededneedupdatesintotoorderbeartifacts.toablepropagatetosupportandthecoordinatethroughputisuserInfrastructure:aservermeansofACSEenvironmentshouldprovide

frameworkfacilities.addingnewtools,languagesandnewforbothCaiseclientprovidesanextensiblecationskindmodel).(e.g.ofdiagrammingtools(e.g.addingarespondingtool)andserverappli-usertreeInisaddition,available.asettotheofwidgetschangingsuchsemanticastheUsability:grammers,Individualfessionalmusthavetools,suchaseditorsanddia-Bothmattingtaskwork-orienteddeveloperswillsufficientbefeaturesthatpro-features,preparedtousethem.features,andavailable.suchsearching,asawarenessandsuchasfor-indicators,teamwork-orientedmustbeuseInactuallyCSEorderweneedtodeterminetothemosteffectivewaystoneering.

takesplacedevelopin(collaborative)anunderstandingsoftwareofwhatengi-Helm,DesignpatternsAlexander’sJohnsonmainconcepts&forsoftwareVlissidesengineering(Gamma,fromthe1995),basedonFiksdahl-King(Alexander,Ishikawa,Silverstein,architecturalJacobson,do-been&Angel1977,Alexander1979)haveeffortsremarkablytoassemblesuccessful.patternlanguagesTheirsuccessforotherhasledfields.

toMartin2004)flecthaveandidentifiedSommervilleanumber(Martinofpatterns&Sommervillewhichre-performthedittasks.waysinExampleswhichgroupsincludeofArtifactpeopleinteractasantodetailtrailandCollaborationinsmallgroupsau-andisavailableathttp://polo.lancs.patterns:furtherInhttp://polo.lanc.pointerananalogousmanner,weare.

workingtoidentifypatternsmodes,Asamorefirstspecificstep,wetoidentifyCSE.

severalinteractionnationeachcharacterisedbythedegreeofcoordi-tivity(taskwork):

required(teamwork)andthenatureoftheac-Private:temporarily,Ausereffectivelywithdrawsfromviabilityothers.ofatypicallychangebeforetoconvincerevealingherselftheitofgroup

thetheitprojectSuchtoaappearusermayfrozenrequiretotheintherestofrathershouldthanbehavingpossibletotorepeatre-integratetime.Ideally,itpublicly.thechangeIndependent:whoseUsersarelocatedinregionsofweakdence.thatsemanticrelationshipsaresufficientlycode

andpendentprojectFrequenttheyintegritycommunicationcansafelyassumeindepen-isnotthreatenedisunnecessaryAusereditingupdates.aGUIAn(view)exampleclassmighttoalterinvolvebyinde-userandyetinteractuserBeditingCaddingacustomerwithotheranewrecord(model)amenu,classclasses.packagewhichdoesnotFollowguidedtheleader:OneusertakesothersersStrictthetour.detailsAnofexampleawouldbetoshowonoth-a

mightWhatifbeusedYoutoSeerecentlycoordinateIsWhatcompletedISee(WYSIWIS)change.inallindividually,amoreusersrelaxedareusingscenario,thesameviews,tool.particularlyHowever,gestures.guidedbyaudiouserscommentarywouldnavigateandAction/Reaction:usersStrongerconstraintsexistas

tictypeterms.becomeForcloserexample,inphysical,ifuserlogicalAchangesorseman-theclassofneedC1athenparameteranotherinuseramethodeditingclassdefinitionC2inmadetothebyupdateausercalls(theactions)tothatmethod.maytoChangesrametersnumberheritanceandandreturntypetypesofclassproperties,aspectssuchthepa-assponsesis(reactions)andinterfaceofmethodsorthein-fromstructureotherwillrequirere-canpotentiallyaffected.Awarenessusersmechanismswhoseworkuseralertuserstopossiblethreats(e.g.anotherportisandandresolvegestures,mechanisms,editingasuperclass).suchastextCollaborationoraudiochannelssup-thecanissues.thenbeemployedtodiscussMˆel´eflictinge:Severalbechangesuserstoareasetmakingofartifacts(potentially-)andthesecon-will

wouldinamediatedtypicallystateoffluxbenegotiatedforaperiod.inadvance,Suchchangesandsuchasanthroughout,audiochannel.byinfrastructurefeatures5

Heuristics

Assoftwareisthecaseforotherkindsofgroupware,andformaintainengineeringingeneral,itisimportanttofrastructureabalancebetweendevelopmentofCSEin-spentandongoingevaluation.Iftoolongistemsthenperformingnotonlydetailedareresultsanalyseslikelyoftoprototypebeofmarginal

sys-usedelayedbutuationoralsomisled.thedevelopmentOntheotherprocessislikelytobeguidedisandtoinformedriskfailurehand,toignoreeval-bybecausedevelopmentisnotuserEvaluatingsuminggroupsonrealisticsystemsproblemsandempiricaltechniqueswork.

isdifficult,withtimetypicaldeveloped.

andexpensive.Variousapproacheshavebeencon-Human-ComputerUsertrialsareparticularlysuitedtoevaluatingpects.oftoolsourThesewouldInterfacebemost(HCI)usefulandtowardsusabilitytheendas-tifyisprojectavailable.whenAtathatfullrangepointofweindustrialwillwishstrengthfeedback/feedthroughsuchfactorsasthebetweenmechanismsrelativemeritstoquan-andofthealternativebalancepotentialthebenefitsofawarenessofothersandtakingsFieldstudiesdistractionsandcasefromstudiesone’sareownlongtasks.

thetermunder-inmustorderconductedtoinrealisticindustrialenvironmentstemsbeinaresupporteddeterminethedomain-specifictaskswhichusedandtoobservehowparticularsys-tiestermsofbothintimepractice.andcostAswelltheyasalsobeingexpensivetheysucharearemostasprovisionusefulwhenofcontrolhavedifficul-thesystemsgroups:consequently,ableatamaturelevel.Inourcase,thesetobewillevaluatedbevalu-userstoexplorethepatternsofcollaborationamongstparticularandthecategorieseffectivenessofindividualtechniquesonitassessingisInimportantordertogaintobetheoftasks.

ablemostfromcostlyevaluationsopment.improveThissystemsallowswhicharetoinaddressearlystagestheissuesofdevel-ofperformance.thesystemthedeveloperstouseresultstouationAratherthansimplyquantifyitsthis.MolichThesetechniquesrangeincludehaveofso-called‘discount’eval-heuristicbeenevaluationdevelopedtoachieveinawhich1990,smallNielsengroups1992,ofevaluatorsNielsen&Landauer(Nielsen1993),&techniquesgivensettherebycanofheuristics.seekviolationsofbeveryeffectiveResultssuggestthatthesedevelopmentenablingthemtobecorrectedindetectingearlierfaults,inthehasAsetcycle.

Gutwinrecentlyofheuristicsforevaluationofgroupwaresummarise2001,beenBaker,proposedGreenberg(Baker,Greenberg&relatetospecificthemCaisehereandfeatures.

indicate&Gutwinbriefly2002).howtheyWeCscwiProvidepropriatetheMeansforIntentionalprovidesExternaltextVerbalCommunication.andCaiseAp-encingandsystems,chatwebcams,suchandanaudiochannel.mayastelephonealsobeused.confer-CscwiiProvidepropriatetheprovidesGesturalMeansCommunicationforIntentionaland.CaiseAp-indicatesausertree(seeFigure3)whichIndividualthelocation(scope)ofeachuser.plementingtoolsfeaturesmaysuchsupplementastelepointers.thisbyim-CscwiiiProvideanConsequentialCommunicationof

plementedIndividual’sviatheEmbodimentusertree..Currentlyim-CscwivProvidetionConsequentialCommunica-Feedthrough)ofSharedsynchronised.Artifacts(i.e.Artifact

toClientreflecttools’changesbufferstoremaintheunderlyingimplementartifacts.fortheageoffeaturesIndividualtoolsmayupdates.suchascolour-codingCscwvProvideicyinCaiseProtectionisto.relyTheondefaultsocialaccessprotocols

pol-whileabilityclientstoundoprovidechanges.

featuressuchastheCscwviManagementCoupledofclientofpanelCollaborationTightlyenableusers.TheandLoosely-tousertreeandtheinterest.potentialusers’conflicts.interests,Feedback,isusedtailoredassesstoalerttoactivitiesusersreflecttoCscwviiAllowCommunicationPeopletoCoordinatebackmechanismschannelssupportcoordination.andTheirotherActionsfeed-.

CscwviiiFacilitatelishingagementContactFindingCollaboratorsandEstab-caterentlywhichtools,.active.users,suchThetoolsasCaiseandtheartifactsusersessiontree,man-areindi-cur-actions,Itisusefultodistinguishtaskwork,task-specificformanceandteamwork,actionsspecifictogroup(CUA)oftasks.CollaborationUsabilityAnalysisper-Greenberg(Pinelle&Gutwin2002,Pinelle,Gutwin&domain-specific2003)providesatechniqueformodellingheuristicevaluation.tasksinordertoformabasisfor5.1

HeuristicsforCSEevaluation

Wementseetems,toheuristicothertechniquesevaluationforasevaluatingavaluableCSEcomple-sys-ityexperiment-drivenassessment—theparticularlyforinfrastructureandcapabil-feedbackareasinwhichwemostdesireposedThePinelleforheuristicsCSCW(Bakerandtasketal.modellingduringdevelopment.techniquespro-what&Gutwin2002,Pinelle2001,etal.Baker2003)etareal.some-2002,mainFirstly,generic.weseeWemeritaimtoinextendestablishingtheseintwoways.manyspecificwaysfromheuristicsthetypicalforCSEsinceadditionalthisdiffersdo-inofCaiseSecondly,logs.weThisadvocateallowsanalysisCSCWapplicationarea.ustomimicandvisualisationmanyofthebeneficialgreatHowever,aspectswecontendofcasestudies.

thatlogscanformance,dealparticular,taskaboutcomplexitycollaborationalsorevealaandmanypatterns,systemper-tensiontheofheuristicstheycanisindicateappropriate,whererefinementotherfactors.orex-InconsideredOurqualitycurrentofsubsequentsetofCSE-specificheuristictherebyevaluations.

improvingheuristics,tocussedforearlier,alongsideappearstheingenericFigureCSCW4.Abriefheuristicsdis-bewiththeCaiseabriefinclusionindicationofeachofheuristicitsrelevanceisgiven,rationaletothetogethercurrentbothHeuristicversion.

ofsets,leadsevaluation,totheidentificationbasedontheandcombinationclassificationoftionsproblemsandissueswithCSE,CSEimplementa-versionAsandantreeasexample,specifictools.

aviolationaproblemidentifiedinourcurrenttheshowsuserlocationofheuristicwell,butCscwdoesiiinotis“Theuserdifficulttransitioncurred.”todecidefromwhatthepreviouschangesinlocation,locationmakingindicateitrentmanticversionSimilarly,asaviolationaproblemidentifiedinhaveourcur-oc-messages.feedbackartifactrepresentationAmetaphorispredominantlyofheuristicCSEvis“Se-wouldmoredeliveredviatextbetightlymorecoupledeffective.”tothe6

VisualisingCSE

Inbasedthisonsection,analysisweofconsiderCaiseeventtheuselogs.ofvisualisationsSuchvisuali-

sationsthingscanprovidevaluableinformationaboutsuchfilesaspatternsofcollaboration,useractivitypro-bothandsequencesofoperationsinrefactoring.ThisofEmpiricalcomplementsdatafromandinformsheuristicevaluations.plete.heuristicslogshelpsensurethatthesetsfiningsetsSuchusedarevalid,representativeandcom-ofdataheuristics.guidesInthereturn,ongoingheuristicsprocesssuggestofre-patternswhichCaisewhichshouldbeobservableineventlogs.eventsrecordsincorporatesdataaboutansuchXML-basedthingsasloggingusers&facilitytools,affected.resultingfromuseractivityandtheartifactsreal-timemulativeinServer-sideordertoobtaintoolsmayinformation,processthesuchlogsascu-inAlternatively,activityindicators,forpropagationtousers.toperformlogsmaybeprocessedoff-lineinordercated)Thenisedvisualisationspotentialdetailedusesanalyses.

inofgroupware(evenrelativelyhavebeenunsophisti-recog-based(Begole,informationtechniquesTangwehave&Hilldeveloped2003).forThesoftwarepipeline-andChurcher,CaiseeventIrwinvisualisation&Cook(Irwin2004)&areChurcherapplicable2002,toFirstly,FigurerequiredXSLT5showslogvisualisation.

orotheratypicalfilterslogselectvisualisationandpipeline.producedered2Ddata.or3DInvisualisationssubsequentstages,whicharelayoutprocesstoolsthespreadsheetforusergraphicsexploration.orvirtualTheseworlds.

maybeasthenvariedren-asXSLXSLEventLogXSLTLayoutFigure5:Visualisinglogdata

cosc.canterbury.ac.nz/dtd/CAISEEventLog.dtdTheeventlogsconformtoaDTD,http://www.enablingcallydataimplementedvalidation,intoXSLT,beperformed.Filters,typi-someWerequiredwillillustrateforspecificextractandformatthethevisualisations.

visualisationprocesswithproximatelyanalysesona30ofminutes,logdatainvolvingfromaCaisefoursessionofap-onFigureprojecttreemaps6showsconsisting(Johnsontwo&viewsoftenusersworkingShneidermanofjavaavisualisationclasses.

1991):inbasedthiscaselogthepipelineendswithoneofSimilarly,eventsistransformedstructuredintoatreerepresentingofourownatools.hierarchyThetypeotherstructuresthus:session(e.g.→componentuser→event→eventtype.shows→siblethatusertwomayusers,readilyNevillebeobtained.andWal,Figure6(a)Figureforproportion6(b)mostprovidesoftheadditionaleventsgeneratedarerespon-informationinthesession.vidualCarlusers’ofactions.eventsofIneachthistyperesultingfromaboutindi-theeventsandWalhaveagreatercase,proportionitcanbeofseenfeedbackthatcansessiondeducethanthattheotherTonyusers.wastheFromleastthisactiveinformationweWalduringwere(inresponsiblefactheleftuserinthisforbeforeitended);Nevilleandcloselythethebulkofthecodingdonewhile(i.e.session;CarlandWalcollaboratedmostforFigureNevilleconcurrently7illustratesandWaleditedthesameartifacts)someworkedmoreindependently.withthenumberafilesamedataset.InthistemporalcasetheanalysispipelineoptionsendsofreadableeventsgeneratedbyExcel.byFiguretheactivity7(a)showsofeach

theCSEiSupportmultipleviewsofartifacts.AgivenJavaclassmayberepresentedindifferent

waysbyindividualclienttoolssuchasatexteditor,foldingeditor,usertreeorUMLclassdiagrammer.Changesmadetotheunderlyingartifactbyanytoolshouldbereflectedappropriatelyineachview.Caisetoolssendupdatedartifactstotheserver.Inreturn,theyreceivesyntaxtreescorrespondingtoartifactswhichhavebeenupdatedbyothers.CSEiiSupportDegreeofInterestbasedfeedback/feedthrough.Centraltosoftwareengineeringac-tivitiessuchasrefactoringandcomprehensionisthenotionoftheneighbourhood(context)ofaparticularcomponentorchange(focus).Theneighbourhoodindicatesthemostrele-vantcomponentstobetakenintoaccountfromtheviewpointofthefocus.Forexample,whenmodifyingamethodtheneighbourhoodmightincludethemethoditself,themeth-odsitinvokes&isinvokedby,itshostclassanditsparentclass.Thisfocus+contextconceptisfamiliarinvisualisation.Onecommonapproachistheuseoffisheye-viewtech-niques(Furnas1986,Sarkar&Brown1994)tode-emphasisefeaturesnotintheneighbour-hoodofthefocus.WhenCaiseclientsupdateartifacts,eventsaregeneratedwhosefociarelocatedatthecorrespondingparsetreenodes.Caisetailorsfeedbackaccordingtotheneighbourhoodofsuchnodes,determinedbythesemanticmodel,usepreferencesandspecificclientcapabilities.CSEiiiSupportfine-grainedintegrity.CSErequiresmorepowerfulapproachesthansimpleCSCW

applicationsinordertoreflectthesemanticandsyntacticstructuresimpliedbythesourcecodeorotherartifacts.Caiseusesparsetreesasthebasisforthesemanticmodelitmaintains.CSEivSupportmultiplephysicalandlogicalgranularities.Physicalgranularitylevelsreflectphys-icalpartitioning(URL,directory,file,line,...)whilelogicalgranularitylevels(package,class,method,block,statement,expression,...)reflectsyntacticstructure.CSEvSupportdeepsyntactic-&semantic-basedawareness&feedback.ThegenericCSCWheuris-ticsaddressissuessuchasnotificationofchangesinthelocationofotherusers.InCSE,itisalsoimportanttobeawareofchangesatasemanticlevel(e.g.methodfoo()hasbeendeletedfromclassBar)oralteredrelationshipsinvolvingcomponentsandusers(e.g.anotheruseriseditingamethodwhichthemethodyouareeditingoverrides).Caiseclientsarenotifiedofchangestothesemanticmodel(includinginferredrelationships)andcanreflecttheseasappropriate.CSEviSupportsemanticrelationships.Updatestoartifactsleadtoindirect,andoftensubtle,

changesinsemanticrelationships(extends,overloads,overrides,calls,uses,...)whichshouldbeindicatedtousers.CSEviiSupportprivateworkandre-integration.Userscanworkagainstasnapshotoftheproject

stateandmakeexperimentalchangeswhichwillnotbeseenbyothers.InCaisethissimplyinvolvesclienttoolstemporarilydetachingfromtheserver.CSEviiiSupportbuildsatdifferenttemporalgranularities.Arapidlyevolvingproject,wheredevel-opersmakeinterleavedchanges,couldpotentiallyspendmuchofitstimeinabrokenstateinwhichmanycomponentsareunabletocompile.CSEsystemsmustaccommodatearti-factsthataretemporarilyun-parsableandprojectsthathaveunresolvedcodereferences.Caisepropagatesmodificationeventstousersdirectlyaccessingthesameartifacts,ensur-ingthattheirviewsaresynchronisedatshorttimescales.Theunderlyingsemanticmodelisupdatedonlywhensyntacticcorrectnessisrestored,sothatonacoarsertimescale,otherusersalwaysbuildagainstacorrectversion.

Figure4:CSEHeuristics

(a)Eventsoriginatedbyeachuser(b)Eventsofeachtypeoriginatedbyeachuser

Figure6:TreemapsshowingeventsinaCaisesession

(a)Alleventsperuser(b)Eventsbytype

Figure7:TemporalanalysisofeventsfromaCaisesession

userduringa100secondintervalaswellastheoveralltotals.Peaksandlullsinactivitycanbeseenclearly.InFigure7(b)theeventsarebrokendownbytype,irrespectiveoftheuserresponsiblefortheirgenera-tion.Againpatternsareevident.MosteventsinthissessionareArtifactevents,sincesignificanttextentryisoccurring;Clienteventsaremainlyassociatedwithlocationchangeswithinartifacts;Changeeventsarisefromsemanticchangessuchasalteredinheritancere-lationshipsandareassociatedwithFeedbackeventswhichalertotheruserstothechanges.

Wal

edit)linesofcodeinatexteditor.Anotherexampleistheprovisionoffeedbackabouttherateofchangeofcomponents.7

Conclusions&furtherwork

C9044

Carl

C8999

C9101

Neville

C10244

C9023

C9058

C8977

Tony

C9108

C8988C9012

Figure8:ArtifactaccessesduringaCaisesessionFinally,Figure8showsthespecificartifacts(Javasourcefiles)modifiedbyeachuserduringthesession.Inthiscase,thepipelineproducesafileforthepop-ulardotlayouttool(Gansner&North1999).Inadynamicversionofthisgraph,edgesareaddedandremovedtoreflectthecurrentsessionstate.

Thevisualisationspresentedinthissection,al-thoughtheycontainconsiderableamountsofusefulinformation,arestillrelativelysimple.Wearede-velopingarangeofmoresophisticatedvisualisations,bothforanalysispurposesandforinclusioninclienttools.Anexampleistheuseofcolourtoindicateat-tributes,suchastimesincelasteditorlastuserto

Inthispaper,wehavedescribedcollaborativesoft-wareengineering,reportedonthecurrentstateofourCaiseimplementationandinvestigatedapproachesforunderstandingandevaluatingCSEapplications.Wehaveconsideredseveralapproachestoevalu-ationofCSCWapplicationsingeneral,andCSEinparticular,andhaveexploredthebenefitsofferedbyheuristicevaluationtechniques.CSE-specificheuris-ticshavebeendevelopedinordertoenhancetheef-fectivenessofheuristicevaluationintheCSEdomain.Thesymbioticrelationshipbetweenheuristicsandempiricaldataobtainedfromeventlogshasbeendeveloped.Inordertoachievethegoalofimprov-ingCSEimplementations,itisnecessarytoemployheuristicswhichaccuratelyreflecttheprocessesandpracticesoccurringinrealisticCSEuse.ProblemsidentifiedusingsuchheuristicsarelikelytodirectlyrelevantintheCSEdomain.However,notallrelevantscenariosarereadilyaccessibletoheuristicevaluators.Analysisandvisualisationofempiricaldataobtainedfromeventlogscomplementsheuristics,highlightingpatternsandissuesatvariouslevelsofdetail.

AsourCaiseimplementationmatures,weexpecttomakegreateruseofbothheuristicevaluationandlogvisualisationtoassessthestateofourcurrentver-sionandtosteerongoingdevelopments.

Ourworkoneventmodellingandloggingispartic-ularlyhelpfulaswerefinetheAPIsfordevelopmentofCaiseclienttools.

WebelievethattheincreasedunderstandingofCSEwhichresultsfromourempiricalworkwillhelpus,andotherCSEresearchers,toimprovethequalityofCSEinfrastructureimplementations.Inaddition,itwillassistinthedesignofmoreeffectiveclienttoolsforsupportingcollaboratingsoftwareengineers.References

Alexander,C.(1979),TheTimelessWayofBuilding,

OxfordUniversityPress.Alexander,C.,Ishikawa,S.,Silverstein,M.,Jacob-son,M.,Fiksdahl-King,I.&Angel,S.(1977),A

Patterntion,OxfordLanguage:UniversityTowns,Press.

Buildings,Construc-Baker,HeuristicK.,Greenberg,mechanicsevaluationS.ofgroupware&Gutwin,basedC.(2001),

ongay,teraction:eds,‘Engineeringofcollaboration,inM.Little&L.theNi-NotesProcECHI2001’,forHuman-ComputerVol.2254ofLectureIn-Toronto,inCanada,Computerpp.Science123–139.,Springer-Verlag,Baker,piricalK.,Greenberg,S.&Gutwin,C.(2002),Em-methodologydevelopmentofaheuristicevaluationinComputer‘Proceedingsforofsharedthe2002workspaceACMconferencegroupware,onPress,pp.96–105.supportedcooperativework’,ACMBar,M.with&CVSFogel,,3rdK.edn,(2003),ParaglyphOpenSourcePress.Devlopment

Beck,EmbraceK.(2001),ChangeExtreme,Addison-Wesley.ProgrammingExplained:

Begole,modeling,J.B.,Tang,‘ProceedingsvisualizationsJ.C.&Hill,andR.applications,(2003),Rhythm

siumACMonPress,Userofpp.interfacethe16th11–20.softwareannualandACMtechnology’,sympo-inChurcher,ploringN.&Cerecke,C.(1996),inIEEEJ.GrundyCSCW&supportM.forsoftwaregroupCRC:engineering,Ex-NewZealand,Press,UniversityApperley,eds,‘OZCHI’96’,pp.62–68.ofWaikato,Hamilton,Churcher,geneousN.,visualisationforce-directedIrwin,W.&Cook,layoutC.(2004),Inhomo-isations,inN.pipeline:ChurcherFrom&C.layoutsalgorithmsChurcher,tovisual-intheeds,‘InVis.auInformation2004:ProceedingsoftheAustralasianofformationConferencesVisualisationinResearchSymposium’,andPracticeVol.inIn-35Zealand.Technology,ACS,Christchurch,NewConradi,forR.&Westfechtel,B.(1998),Comput.softwareSurv.configuration30(2),232–282.management’,‘Versionmodels

ACMCook,workC.&D.forChurcher,collaborativeN.(2003),softwareAnextensibleframe-theAzada,ference’,10thAsia-Pacificed.,‘APSEC2003:engineering,Proceedingsinofpp.290–299.IEEEPress,SoftwareChiangEngineeringMai,Thailand,Con-Cook,synchronousC.,Irwin,W.&Churcher,N.(2004),incollaborativesoftwareengineering,Towards

Engineering‘ProcAPSEC2004:Conference’,11thBusan,AsiaPacificKorea.SoftwareDommel,‘FloorH.-P.&collaboration’,controlGarcia-Luna-Aceves,forJ.J.(1997),

MultimediamultimediaSyst.conferencing5(1),23–38.andEllis,SomeC.,Gibbs,S.&Rein,G.(1991),‘Groupware:

oftheissuesACM34and(1),experiences’,38–58.CommunicationsFurnas,ACMG.inComputingSIGCHI(1986),Generalised’86Systems’,Conferencefisheyepp.16–23.onHumanviews,inFactors‘Proc

Gamma,(1995),E.,Helm,R.,Johnson,R.Object-OrientedDesignPatterns:&Vlissides,J.

Software,ElementsAddison-Wesley.

ofReusableGansner,visualizationE.R.&North,S.C.(1999),‘Anopengraph

waresystemanditsapplicationstosoft-rienceengineering’,30(11),1203–1233.Software—PracticeandExpe-Greenberg,Computer-SupportedS.(1989),TheReport,Cooperative1988ConferenceWork:Tripon

pp.Intelligence,49–55.in‘SIGCHIAlsoBulletin’,Vol.20of5,ACM,19,publishedApril1989.inCanadianArtificialGrudin,ProblemsJ.(1992),ganizationalintheWhyDesignCSCWandApplicationsEvaluationofFail:

Or-eds,Interfaces,inD.Marca&G.Supported‘Groupware:SoftwareforComputer-Bock,Alamitos,CA,Cooperativepp.552–560.Work’,IEEEPress,LosGrudin,EightJ.(1994),Groupwarecationschallengespp.92–105.oftheACM’,fordevelopers,andsocialVol.37of1in,ACM‘Communi-dynamics:

Press,Irwin,sationW.&H.ferencesYan,pipeline,Churcher,eds,‘VisualisationinN.D.(2002),D.Feng,XMLinthevisuali-2001’,J.Jin,Vol.P.11Eadesof&TechnologyinSelected,ResearchACS,Sydney,andPracticeinInformationCon-shoponVisualpapersInformationfrom2001Australia,Processing.Pan-Sydneypp.59–68.Work-Irwin,metrics:W.&isations,PrecisionChurcher,toolsN.(2003),andObjectoriented

siumAustralia,onSoftwarein‘METRICS2003:configurablepp.112–123.Metrics’,IEEE9thPress,IEEESympo-visual-Sydney,Johnson,AB.&Shneiderman,B.(1991),Tree-maps:

hierarchicalspace-fillingsoninformationapproachstructures,tothevisualizationinG.Niel-of’91’,&tos,CA,IEEEL.Rosenblum,pp.Computer284–291.Societyeds,‘proc.Press,VisializationLosAlami-Martin,cooperativeD.&Sommerville,I.(2004),‘Patternsof

ologyInteract.andinteraction:Linkingethnomethod-11design’,(1),59–89.ACMTrans.Comput.-Hum.Munson,controlJ.P.in‘Computerframework&Dewan,P.(1996),Aconcurrency

SupportedforcollaborativeCooperativesystems,Work’,pp.278–287.Nielsen,throughJ.(1992),thecomputingSIGCHIheuristicsystems’,conferenceevaluation,Findingusabilityproblems

ACMonin‘ProceedingsofPress,Humanpp.373–380.factorsinNielsen,cal‘ProceedingsmodelJ.&Landauer,ofthefindingT.K.of(1993),usabilityAmathemati-manpp.206–213.factorsinofcomputingtheSIGCHIsystems’,conferenceproblems,ACMonPress,Hu-inNielsen,ofJ.&Molich,R.(1990),Heuristicevaluation

conferenceuserinterfaces,tems’,ACMonPress,Humanin‘Proceedingspp.factors249–256.incomputingoftheSIGCHIsys-Pinelle,through:D.&Gutwin,C.(2002),evaluation,addingferencein‘ProceedingscontexttoGroupwarewalk-ofgroupwaretheusabilityACMPress,onHumanpp.455–462.

factorsincomputingSIGCHIsystems’,con-Pinelle,D.,Gutwin,C.&Greenberg,S.(2003),

‘Taskanalysisforgroupwareusabilityevalu-ation:Modelingshared-workspacetaskswiththemechanicsofcollaboration’,ACMTrans.Comput.-Hum.Interact.10(4),281–311.Roseman,M.&Greenberg,S.(1996),‘Buildingreal-timegroupwarewithGroupKit,agroupwaretoolkit’,ACMTrans.Computer-HumanInter-action3(1),66–106.Sarkar,M.&Brown,M.(1994),‘Graphicalfisheye

views’,CommunicationsoftheACM37(12),73–84.Sarma,A.&vanderHoek,A.(2002),

Coordinatingdistributedworkspaces,COMPSAC2002:26thComputerandApplicationsConference’,Oxford,pp.1093–1097.

Palant´ır:in‘ProcSoftwareEngland,

Sarma,A.,Noroozi,Z.&vanderHoek,A.

(2003),Palant´ır:Raisingawarenessamongcon-figurationmanagementworkspaces,in‘ProcICSE2003:25thInternationalConferenceonSoftwareEngineering’,Portland,Oregon,pp.444–454.Schummer,T.(2001),LostandFoundinSoftware

Space,in‘34thAnnualHawaiiInternationalConferenceonSystemSciences’,IEEE,Maui,Hawaii.SourceForge(2004),‘SourceForge.netHomePage’,

http://sourceforge.net/.Tvedt,R.T.,Costa,P.&Lindvall,M.(2004),Eval-uatingsoftwarearchitectures,inM.Zelkowitz,ed.,‘ArchitecturalIssues’,Vol.61ofAdvancesinComputers,AcademicPress,NewYork,pp.2–43.Twidale,M.&Nichols,D.(2004),Usabilitydiscus-sionsinopensourcedevelopment,WorkingPa-per08/2004,DepartmentofComputerScience,UniversityofWaikato,Hamilton,NewZealand.