First experience was gained regarding the use of theGQMapproach for the practical evaluation of an expe-rience base by its successful application to the evalua-tion of the experience base CBR-PEB.
Data on real use is being collected. When an appropri-ate amount of data has been collected, another feed-back session with this real data and an iteration of theGQMprocess will be carried out.
We are also planning to develop a comprehensive eval-uation technique for experience bases and test and val-idate this with our experience bases.
7 References
[AABM95]K.-D. Althoff, E.Auriol, R.Barletta, andM.Manago.A Review of Industrial Case-Based ReasoningTools. AI Intelligence. Oxford (UK), 1995.
[ABS96]Klaus-Dieter Althoff and Brigitte Bartsch-Spörl.Decision support for case-based applications.Wirtschaftsin-formatik, 38(1):8–16, February 1996.
[ABT97]K.-D. Althoff, A.Birk, and C.Tautz. The experi-ence factory approach: Realizing learning from experiencein software development organizations. InProceedings ofthe Tenth German Workshop on Machine Learning, Univer-sity of Karlsruhe, 1997.
[Alt97]Klaus-Dieter Althoff. Evaluating case-based reason-ing systems: The Inreca case study. Postdoctoral thesis (Ha-bilitationsschrift), University of Kaiserslautern, 1997.
[AP94]Agnar Aamodt and Enric Plaza. Case-based reason-ing: Foundational issues, methodological variations, andsystem approaches.AICom - Artificial Intelligence Commu-nications, 7(1):39–59, March 1994.
[AW97]Klaus-Dieter Althoff and Wolfgang Wilke. Potentialuses of case-based reasoning in experienced based construc-tion of software systems and business process support. InR.Bergmann and W.Wilke, editors,Proceedings of theFifth German Workshop on Case-Based Reasoning,LSA-97-01E, pages 31–38. Centre for Learning Systemsand Applications, University of Kaiserslautern, March 1997. [BA98]R.Bergmann and K.-D. Althoff. Methodology forbuilding case-based reasoning applications. In M.Lenz,B.Bartsch-Spörl, H.D. Burkhard, and S.Wess, editors,Case-Based Reasoning Technology - From Foundations toApplications, pages 299–328. Springer Verlag, 1998.
[BCR94a]VictorR. Basili, Gianluigi Caldiera, and H.DieterRombach. Experience Factory. In JohnJ. Marciniak, editor,Encyclopedia of Software Engineering, volume1, pages469–476. John Wiley & Sons, 1994.
[BCR94b]VictorR. Basili, Gianluigi Caldiera, and H.DieterRombach. Goal Question Metric Paradigm. In JohnJ. Mar-ciniak, editor,Encyclopedia of Software Engineering,volume1, pages 528–532. John Wiley & Sons, 1994.
[BDR96]LionelC. Briand, ChristianeM. Differding, andH.Dieter Rombach. Practical guidelines for measure-ment-based process improvement.Software Process,2(4):253–280, December 1996.
[BR91]VictorR. Basili and H.Dieter Rombach. Support forcomprehensive reuse.IEEE Software Engineering Journal,6(5):303–316, September 1991.
[BSAM97]Brigitte Bartsch-Spörl, Klaus-Dieter Althoff, and
1.http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html
Alexandre Meissonnier. Learning from and reasoning aboutcase-based reasoning systems. InProceedings of the FourthGerman Conference on Knowledge-Based Systems (XPS97),March 1997.
[CEM96]The CEMP Consortium. Customized establish-ment of measurement programs. Final report, ESSI ProjectNr. 10358, Germany, 1996.
[Coo97]WilliamS. Cooper. On selecting a measure of re-trieval effectiveness. In K.S. Jones and P.Willet, editors,Readings in Information Retrieval, pages 191–204. MorganKaufmann Publishers, 1997.
[ELS97]Robert Engels, Guido Lindner, and Rudi Studer. Aguided tour through the data mining jungle. InProceedingsof the Tenth German Workshop on Machine Learning, Au-gust 1997.
[ELS98]Robert Engels, Guido Lindner, and Rudi Studer.Providing user support for developing knowledge discoveryapplications: A midterm report.Künstliche Intelligenz,1:40–45, 1998.
[GFW94]MartinL. Griss, John Favaro, and Paul Walton.Managerial and organizational issues - starting and runninga software reuse program. In W.Schäfer, R.Prieto-Diaz,and M.Matsumoto, editors,Software Reusability, chapter3,pages 51–78. Ellis Horwood Ltd., 1994.
[GHW95]Christiane Gresse, Barbara Hoisl, and JürgenWüst. A process model for GQM-based measurement. Tech-nical Report STTI-95-04-E, Software Technologie TransferInitiative Kaiserslautern, Fachbereich Informatik, Univer-sität Kaiserslautern, D-67653 Kaiserslautern, 1995. [Hae95]Theo Haerder.Skript zur Vorlesung Datenbanksys-teme. AG Datenbanksysteme, Universität Kaiserslautern,1995.
[HR97]Susanne Hartkopf and Günther Ruhe. Goal-orientedlearning in experimental software engineering by usingrough sets. InProceedings of the Tenth German Workshopon Machine Learning, August 1997.
[Icm98]The Methodology of Applying Machine Learning.AAAI/ICML 1998 Workshop, Madison (WI), July 1998. [MBC+94]M.Manago, R.Bergmann, N.Conruyt,R.Traphöner, J.Pasley, J.LeRenard, F.Maurer, S.Wess,K.-D. Althoff, and S.Dumont. Casuel: A common case rep-resentation language. Technical Report Deliverable D1,Version 2.01, Esprit Project Inreca (P6322), 1994. [Mei96]Alexandre Meissonnier. A case-based informationsystem for case-based applications and systems based on IN-RECA (in German). Diploma thesis, AI/Expert SystemGroup, University of Kaiserslautern, October 1996. [Nic98]Markus Nick. Implementation and Evaluation of anExperience Base. Diploma thesis, Fraunhofer IESE, Univer-sity of Kaiserslautern, 1998.
[RIG93]RIG Glossary. Technical Report RTR-0001, ReuseLibrary Interoperability Group (RIG), April 1993. [TA97]Carsten Tautz and Klaus-Dieter Althoff. Usingcase-based reasoning for reusing software knowledge. InD.Leake and E.Plaza, editors,Proceedings of the SecondInternational Conference on Case-Based Reasoning.Springer Verlag, 1997.
[Wes95]S.Wess.Case-Based Reasoning in Knowl-edge-Based Systems for Decision Support and Diagnostics(in German). PhD thesis, University of Kaiserslautern, 1995;also: infix Verlag, Sankt Augustin.
Figure5).
We propose that making on-line updating possiblewill also result in cases that are more up-to-datebecause the contributors can easily update the infor-mation. So, the whole system will be moreup-to-date than, e.g., a book can ever be. This isfundamental because up-to-date information wasregarded as very important in the evaluation pro-gram (see Section5).•Removal of cases:
Cases can be removed by the EF staff on request orif the evaluation program shows that a case is con-sidered “not useful” by a certain percentage ofusers.
5 Evaluating an Experience Factory/Base5.1 Choosing the Evaluation Focus
Basili, Caldiera, and Rombach propose that softwaredevelopment must be seen and treated as a business.As already stated, an experience factory is establishedto improve this business by supporting and improvingreuse [BCR94b]. Since the obviousultimate goal of abusiness issuccess – especially economic success –, anevaluation of an experience factory and its experiencebase must focus on its success. Then, such an evalua-tion can be used to justify the establishment of anexperience factory.
5.2 Choosing the Evaluation Technique
For the evaluation a standard technique of the Fraun-hofer IESE and the AG Software Engineering of theUniversity of Kaiserslautern for goal-oriented mea-surement and evaluation – the Goal/Question/Metric(GQM) approach [BCR94b, GHW95, BDR96, HR97]– is used.
GQM is an innovative technology for goal-orientedsoftware engineering measurement. GQM helps defin-ing and implementing operational and measurablesoftware improvement goals. It has been successfullyapplied in several companies, such as NASA-SEL,Bosch, Digital, and Schlumberger[CEM96]. In GQMprograms, the analysis task of measurement is speci-fied precisely and explicitly by detailed measurementgoals, called GQM goals. Relevant measures arederived in a top-down fashion based on the goals via aset of questions and quality/resource models. Thisrefinement is precisely documented in a GQM plan,providing an explicit rationale for the selection of theunderlying measures. The data collected is interpretedin a bottom-up fashion considering the limitations andassumptions underlying each measure.
To create an effective measurement plan, a processmodel is needed to assign the measures to elementary
processes and products. The software developmentmodel for CBR-PEB (see Section4.1.2) is such a pro-cess model.
5.3 Practical Evaluation
Domain experts were interviewed in order to maketheir implicit knowledge about success criteria ofCBRsystems explicit, which led to detailed measure-ment criteria for CBR-PEB (from goals to questions tomeasures). The measurement criteria focus on the util-ity of the information supplied by CBR-PEB and theuser friendliness of its WWW interface. Analyses ofthe measures showed that the success is closely relatedto the usability of an experience base as defined by[RIG93] and the effectiveness of an informationretrieval system as defined by [Coo97]. The utilitymeasures put an emphasis on the quality of theretrieved information and also helped identifying theimportant attributes of the characterization scheme andtheir proposed interrelationships, which can beregarded as general knowledge about the domain thatis being validated by the GQM process.
The interpretation of the results of the measurementprogram takes place in well-structured, organizedmeetings, the so-calledfeedback sessions.
5.4 Preliminary Results
In order to make a first validation of the measurementprogram, usage trials with the experts were conducted.These usage trials not only supplied measurement datafor the first feedback session, but also showed how dif-ferent users actually interact with the system, and gavehints for improvements of the system.
In the first feedback session, the system and the mea-surement program received a very positive rating bythe experts. They also stated that the system at its cur-rent state is already useful (that was not expected bythe experts in the beginning) and, therefore, should beinstalled as soon as possible.
The first iteration of the GQM process for CBR-PEBalso showed that the GQM approach is useful for eval-uation of an experience base and led to a meaningfulresult.
6 Summary and Future Work
A comprehensive model for the experience factoryconsisting of an organizational structure and a soft-ware development model was presented. The applica-tion of CBR technology in this context was shown. Ageneric architecture for an experience factory toolusing CBR technology was described.
The experience factory model and the architecturewere successfully instantiated for CBR-PEB - an expe-rience base for CBR-system development. CBR-PEB
Project organization's tasks.A project organiza-tion (“user” of CBR-PEB) develops a CBR system asfollows:
First the project organization collects the decision sup-port criteria considering the CBR system the organiza-tion wants to develop. The criteria are divided intodomain, task, ergonomic, and technical criteria (see[Mei96] and [ABS96] for further information or have alook at the questionnaire in the WWW1). In order toreceive decision support the user retrieves the mostsimilar cases from the CBR-PEB case base accordingto the collected criteria. CBR-PEB lists the ten mostsimilar cases together with their similarity rating.2 Theuser can browse through these cases to carry out fur-ther subjective investigations. Based on these evalua-tions3 he decides whether he wants to reuse an existingsystem or knowledge, and, if so, which system (appli-cation or tool) he wants to use as a starting point for hisdevelopment efforts or which knowledge from thedevelopment of existing systems he wants to reuse.After making the decision he fetches or buys the exist-ing system or knowledge (i.e., the existing solution).Then he develops his new system reusing the existingsystem or knowledge. Afterwards he answers theWWW questionnaire for CBR system development inorder to provide a new case for CBR-PEB.
A CBR system developer who does not reuse existingwork, but is willing to share his experience with oth-ers, may also answer the WWW questionnaire.Note that there is only one single reuse process consid-ering CBR-PEB per CBR system development.When a case should be updated (e.g., when a CBR sys-tem is improved), the developer provides the informa-tion for the update of the case to CBR-PEB (this hasnot been explicitly modeled in Figure5).
Experience factory's tasks.The case base ofCBR-PEB [Mei96] contains experience packages thatdescribe CBR systems (i.e., software products). Thereare two subtypes of CBR system packages: CBR appli-cation and CBR tool packages.
Before storing a new CBR system package inCBR-PEB's case base the new information must bechecked to see if it is valuable and complete or justnonsense input from a surfer. Further inquiring, if nec-essary, and checking and storing is done by the Fraun-hofer IESE. This is the same for update informationregarding existing cases.
1.http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html2.The system uses local type-specific and global similarity mea-sures and weighting. For more in-depth informationsee[Wes95].
3.automatic evaluation by CBR-PEB and/or personal and subjec-tive evaluation by the project organization
TCP/IP socketdata storage layerCBR-WorksUNIX file systemCBR-PEBcontactcase basemeasurementaddressCBR-systemdata filedeveloperFigure 6:Data storage layer of CBR-PEB.At the moment, the case base contains 45cases. Anoverview of the content of the case base is givenin[BSAM97].
4.2 Tailoring the Generic Architecture forCBR-PEB
The generic architecture was simplified for CBR-PEBat two points in the data storage layer (see Figure6):1.CBR-PEB does not contain any experience pack-ages, i.e., a CBR system. It only consists of thecase base as an index and a contact address in eachcase that models the reference.
2.The measurement data base is implemented as asimple file using the system-library file-accessfunctions.
CBR-Works fromTECINNO GmbH4 (Kaiserslautern,Germany) is used as CBR tool. CBR-Works is queriedby the EFserver over a TCP/IP socket using theCQL/Casuel interface [MBC+94].
4.3 The Features of CBR-PEB
The system implements all features listed inSection2.2, Table52. The WWW interface ofCBR-PEB especially supports the querying, creating,and updating of cases. As already stated, in the reusestep the system can only provide a contact address as areference but not the system that shall be reused.
The features dealing with learning (i.e., recording andpackaging) are especially worthy of note:•Contribution of new cases:
The new cases are reviewed by the EF staff in orderto filter nonsense input from net surfers.•Update of existing cases:
The case contributors can update their cases on-line.The authorization uses a login name and a pass-word. The changes are written to a log file to allowthe EF staff to undo an unauthorized or invalidrepackaging.
This updating by the case contributor is regarded asa recording of update infos and repackaging of theexisting case (this is not explicitly modeled in
4.http://www.tecinno.de/
5.http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html
4.1.1 Organizational Structure
The organizational structure in the context ofCBR-PEB as shown in Figure4 has been derived fromthe organizational structure in the generic model.
software business companyInterNet, etc.experience existing inthe worldat largeproject organizationCBR systemdevelopmentprocessreuse process•The only link between the CBR-PEB EF and theproject organizations (i.e., the users of CBR-PEB) isthe Internet.•There is no parent organization for all project orga-nizations, who use CBR-PEB, like the developmentorganization in the generic model.•There is no organization like the management in thegeneric model that has an influence on both theexperience factory and the project organizations.•The reuse of CBR-PEB experience takes place onlyonce at the beginning of the project (see develop-ment process description below).•Each software business company may also have itsown experience factory as described in the genericmodel. So the CBR-PEB experience factory servesas an additional external source of experience thatcan be queried by the project organizations directlyor by the company's EF depending on the com-pany's organizational policies.Note that the difference between using CBR-PEB andinfusing “experience existing in the world at large” isthat there is a closed loop with feedback forCBR-PEB.
In the future, the generic model could also be extendedwith the concept of external experience factories. Thisis the typical role the Fraunhofer IESE plays whenestablishing an EF at a partner organization.
manage-mentEFreuseInterNetrecordexperience factory (Fraunhofer IESE)experience base CBR-PEBinformation flowoptional inf. flowinfluence
Figure 4:The organizational structure withCBR-PEB.The differences with respect to the generic model areas follows:
•CBR-PEB is an experience base outside the com-pany.
•Not only one but several companies can accessCBR-PEB.
criteria(domain, task, technical,ergonomic)develop CBR systemwithout reuse4.1.2Software Development Model
The software development model is shown in Figure5.This model has been derived from the generic modeland is tailored to the needs of CBR-PEB, i.e.,CBR-system development. The model also coincideswith the CBR-PEB usage scenario as described by thecreator of CBR-PEB [Mei96]. In the following themodel will be described.
required CBR systemlessons learned, etc.develop CBR systemwith reuseknowledgeexisting CBR systemselectnew CBRsystem infoscontribute new case(fill out questionnaire)development process modelidentifyreuse process modelproject organizationexperience factory(IESE)CBR system packagecreateCBR-PEBCBR appl package(re-)packageCBR tool packagerecord(check+store)Figure 5:The software development model for CBR systems with CBR-PEB.
cases from the case base. Then the solution fromone of the cases is reused and adapted, i.e., modifiedto fulfill the reuse requirements. The result is therequired object, the solution to the problem. Theapplication of the solution, which can be comparedwith integration, is not explicitly part of the CBRcycle. The revise and retain steps are similar torecording and packaging.CBRcycleexperiencefactorysupport bya CBR toolretrieveidentify,queryingevaluate + selectreusemodify or createretrieve experience pack-agearevise +record, (re-)pack-create/update/ removeretainagecases, update similarity andgeneral knowledgeTable 2:How the CBR cycle matches the re-use-oriented software developmentmodel.
a.The “experience package” is the actual reuse candidate,which is modified to fulfill the reuse requirements.
Thus, the minimum set of features of a CBR tool for anexperience base are
•querying the case base (similarity-based retrieval),•retrieval of the experience package that is character-ized by the selected case,•adding a new case to the case base,•modifying an existing case in the case base,•removing an existing case from the case base.For each of these actions related to recording andpackaging, criteria that trigger these actions should bedeveloped. An experience base tool should support themaintenance by checking the criteria and/or triggeringthe actions.
3 A Generic Architecture for the Realizationof an Experience Base
The basic system architecture – as shown in Figure3 –has been derived from today’s typical architecture forDB-oriented client-server applications [Hae95] con-sisting of three logical layers: presentation, applica-tion, and data storage layer. Here, the presentation isdone by a WWW browser (e.g., Netscape Navigator).The application is represented by the EFserver forquerying the case base and on-line data collection. Thedata storage layer consists of the data bases and their
DBMSs1 (i.e., a CBRtool for the case base and anappropriate DBMS for the experience package DB).
WWW browserWWW browserHTTPpresentation layerapplication layerEF serverdata storage layerCBR toolDBMSDBMScase baserefEP DBmeasurement DBFigure 3:Experience factory tool architecture.The EF server must be installed at a WWW server orinclude one. The components from the data storagelayer can be installed at the same computer system orcan be distributed over the network.
The storing of the experience packages is managed bya separate DBMS. The CBR tool provides only anindex for the experience package DB. Each case con-tains a reference to the experience package it describes(if an experience package is not classified then therewill not be any case describing it).
This implies that the EFserver manages the distribu-tion of the EB data across the case base and the experi-ence package DB (EPDB).
Of course, the same DBMS or a single DBMS may beused for measurement DB and experience packageDB.
4 CBR-PEB – An Experience Base forCBR-System Development
4.1 Tailoring the Generic Model for CBR-PEB
As already stated in section2.2, a case base likeCBR-PEB can be viewed as an experience base. Expe-rience from CBR-PEB can be reused for CBR-systemdevelopment. The (re-)users (i.e., CBR-system devel-opers) can give feedback on CBR-system developmentto the CBR-PEB maintenance staff at the FraunhoferIESE by filling out the form/questionnaire with infor-mation about their new system and keeping this infor-mation up-to-date. The maintenance staff acts as anexperience factory that maintains the experience baseCBR-PEB. Thus, the generic model as presented inSection2.1 can be tailored to the needs of CBR-PEB.In the following the coincidences, if not obvious, andthe differences concerning organizational structure andsoftware development model are described.
1.DBMS = Data Base Management System
development process model (j)experiencecreateexistingxin the worldkxkat largeidentifymodify“world”Aikevaluate andselectreuse process model (k)transfer intoprojectorganizationalorganization ownershiprecord(“infuse”)project-specificexperiencefactoryexperienceAikA1ikmexperience basecreate(re-)package (l)Figure 2:The reuse-oriented software develop-ment model.requirements are specified for objects that shall beintegrated. For each object a reuse attemptk may takeplace. A reuse attempt runs as follows:
Given the project-specific reuse requirementsxk for anobject, a set of reuse candidates
Aik,…,Aik from
1
m
the experience base is identified. Their potential to sat-isfy the reuse requirementsxk is evaluated, and thebest-suited candidate
Aik is selected. Then either this
candidate is modified to the needs of the project, ifrequired, or a new object is developed, if reuse is notconsidered to be appropriate. The resulting object iscalledxk.
The reuse attempt may fail during identification, selec-tion, modification, or integration. Failure means thatthe reuse process is aborted and then either, instead ofreuse, the required objectxk is created from scratch ora new attempt is started.
Table1 gives an overview of the input and output ofthe sub-processes of the reuse process.
Experience factory's tasks.In the beginning theexperience base must be created. This includes choos-ing or developing a tool, developing characterizationschemes, and implementing the experience base.During operation a new object, if modification or cre-ation took place during development, and/or experi-ence gained in the reuse or creation (lessons learned,
processinputoutputidentifyxk,EBAik,…,A1ikmevaluateand selectxk;Aik,…,AikA1mikmodifyxk;AikxkTable 1:Input and output of the elementary re-use processesetc.) may be recorded in the experience base. Record-ing includes checking whether the object and/or expe-rience is worthwhile to store and, if so, storing the newexperience in the experience base.Also experience from outside the organization (e.g.,commercial off-the-shelf (COTS) software) may beinfused into the experience base.To increase its reuse potential, experience can be(re-)packaged (i.e., tailored, generalized, or formal-ized) [BR91].1 Another way to repackage the experi-ence is to change the characterization scheme or torework the relations among artifacts (e.g., dependen-cies, variations, specializations). Repackaging alsoincludes changing the tool used to store the experi-ence, e.g., choosing a more advanced tool or improv-ing retrieval mechanisms. Note that a more advancedtool usually requires a different representation, thusadditional repackaging of the experience base withrespect to characterization scheme and contentsbecomes necessary.
2.2 Using CBR in the Experience Factory
The reuse-oriented software development model,which is part of the generic model, is quite similar tothe CBRcycle, a common process model forcase-based reasoning (see [AP94]). In fact CBR tech-nology can be used to support reuse in the experiencefactory:2
•The case base can be seen as an experience base,and each case as an experience package. For exam-ple, the CBR system CBR-PEB includes such a casebase.
•The similarities between the processes are as fol-lows (see Table2 for an overview). The reuserequirements describe the problem. This problemdescription is used to retrieve one or more similar
1.“Packaging” refers to the first tailoring, generalizing, or formal-izing of an artifact, “repackaging” to any additional tailoring,generalizing, or formalizing of the same artifact.
2.A more detailed view using the CBR task-method decomposi-tion model, a refinement of the CBR cycle, is presented in[TA97].
ence base. Section4 describes the instantiation of theexperience factory model and tool architecture forCBR-PEB. Section5 summarizes the evaluation andits current status. Section6 gives a summary and anoverview of future work.
2 Support for Reuse and Learning in theExperience Factory
“A continuous build-up of knowledge requires an orga-nizational structure” [ABT97]. In order to operational-ize reuse and learning and provide tool support for it, aprocess model is required. In [BR91] such a processmodel for reuse-oriented software development is pre-sented. In this section, the experience factory as anorganizational framework for reuse and learning isintegrated with the reuse-oriented software develop-ment model. The relation of the reuse-oriented soft-ware development model to the basic CBR cycle ismade explicit.
This shows how CBR is (and can be) used as a tech-nology to support the reuse and learning processes.Furthermore the required features for an experiencefactory tool are identified. For such features wheresupport through machine learning technology is use-ful.
CBR-PEB is a concrete example that shows that ourapproach is applicable. Since there are some differ-ences between CBR-PEB and the generic model forreuse and learning in the experience factory and, inaddition, it is easier to understand on the generic levelof description, it is worthwhile to know the genericmodel as well as its tailoring for CBR-PEB.
2.1 Generic Model
The generic model is based on the experience factoryparadigm and its comprehensive reuse mechanisms,also called reuse-oriented software development model[BR91]. The model focuses on the reuse process itself.Neither the project development with reuse is explic-itly modeled nor the development for reuse.
The generic model is presented in two parts. The orga-nizational structure shows the relation between organi-zational unit – especially the information flow – andthe number of instances for each process and eachorganizational unit in a typical software business com-pany. The software development model, a slightlymodified version of the reuse-oriented software devel-opment model [BR91], describes the process modelsconcerning software development with reuse and expe-rience base maintenance.
software business companydevelopment organizationproject organizationdevelopmentprocessmanage-reuse processmentreuseexperiencerecordexisting inthe worldexperience factoryat largeinfuseexperience baseinformation flow
optional information flowinfluence
Figure 1:The organizational structure of a typicalsoftware business company.
2.1.1 Organizational Structure
A typical software business company consists of adevelopment organization, the EF organization, andthe management (see Figure1).
The development organization consists of severalproject organizations. Each of them is developing orhas developed the deliverables of a project. During theproject development several reuse attempts1 (i.e.,instances of the reuse process model) may take place.The experience factory organization aims at supportingreuse of life-cycle products and related experience(such as guidance for their development). Therefore, itcreates and maintains the experience base and per-forms certain tasks involved in reuse and/or providestechnology for the performance of these tasks.The management of the business company has aninfluence on both the development organization andthe experience factory. Thus, the management can helpintroduce a reuse program by, e.g., rewards, incentives,financing an EF, or other forms of commitment.
2.1.2 Software Development Model
To reflect our needs, the reuse-oriented software devel-opment model [BR91] is extended with the process'createEB' and processes are associated with the orga-nizational units (i.e., experience factory and projectorganization) – see Figure2.
Project organization's tasks.The project organiza-tion develops project deliverables. During a project
1.“Attempt” refers more to the “experimental” nature of the reuseprocess than the word “process” because reuse may fail.
Concepts for Reuse in the Experience Factory andTheir Implementation for CBR-System Development
Klaus-Dieter Althoff, Markus Nick, Carsten Tautz
Fraunhofer Institute for Experimental Software Engineering (Fraunhofer IESE)Sauerwiesen 6, D-67661 Kaiserslautern, Germanyemail: {althoff, nick, tautz}@iese.fhg.de
Abstract.An Experience Factory is an infrastructure for organizational learning in software develop-ment that includes an Experience Base as an organizational memory. We introduce a system architecturehow such an infrastructure can be technically supported based on Case-Based Reasoning (CBR) technol-ogy. As a first instantiation of this architecture we present the CBR-PEB application, a publicly accessi-ble WWW-based experience base for CBR-system development. Based on first experiments, someresults about the evaluation of the success of this application are described.
Keywords.Case-based reasoning, continuous learning from experience, experimental software engi-neering, experience factory, experience base, GQM-based measurement
1 Introduction
In software development, reuse is regarded as a meansto handle today's increasing quality, productivity, andtime-to-market requirements [BCR94a, GFW94]. Butreuse is not just a by-product of software development.To support reuse one has to establish an infrastructureaimed at capitalization and reuse of life cycle experi-ence and products [GFW94, p62]. Basili and Rombach[BCR94a] call such an infrastructureexperience fac-tory(EF).
The central element of the experience factory is theexperience base in which the experience is stored inexperience packages [BCR94a]. Griss et al. [GFW94]report that simple lists – like World-Wide-Web(WWW/Web) pages – can be used for a small experi-ence base of 30–50 components, and for a largerlibrary of 100–200 parts, a set of well organized andindexed catalogs has been used effectively. Beyondthat, more powerful tools might be necessary.In [TA97, ABT97] it is shown thatCase-Based Rea-soning (CBR) is a very promising technology for therealization of an experience base, i.e., techniques,methods, and tools from the domain of CBR can beuseful for building and maintaining an experiencebase.
At the Fraunhofer IESE we are using CBR technologyto develop experience bases not only for research pur-poses but also for industrial use. Thus, we are applyingmachine learning technology to real-life problems. Inaddition, the experience factory approach is very use-ful for operationalizing learning systems and/orknowledge-based systems in industrial/business envi-ronments [AW97, ABT97, BA98]. This might also behelpful in the context of building methodologies forapplying data mining (e.g., [ELS97, ELS98]) andmachine learning in general [Icm98].
One focus of this paper is on describing a concreteexperience base that has been developed for supportingCBR-system development. The emphasis is put onproviding decision support for reusing existing CBRsystems for the development of new systems. In orderto make the system easily accessible by CBR develop-ers all around the world, the system has been madepublicly available over the WWW. By this, a servicefor the whole CBR community is offered at no charge.Maybe for other dynamic subfields of machine learn-ing (e.g., data mining) such a kind of service would behelpful, too.
This experience base is based on a number of researchefforts: [Alt97, ABS96, AABM95] developed the clas-sification criteria for CBR systems. [BSAM97, Mei96]conducted a survey about CBR systems and madethese experiences reusable by means of CBR technol-ogy, i.e., each questionnaire has been represented as astructured case. As a part of [Nic98], this system hasbeen made publicly accessible over the WWW and anevaluation program was developed in order to show theusefulness of the system from the viewpoint of itsusers.
The rest of the paper is structured as follows. Section2describes how the experience factory as an organiza-tional framework together with a process model forreuse and learning can support reuse and learning in asoftware business company and shows its relation tothe basic CBR cycle (introduced by Aamodt and Plaza[AP94]) including the identification of points for toolsupport. Section3 introduces a generic architecture foran experience factory tool using CBR technology thatserves as a basis for the implementation of an experi-
因篇幅问题不能全部显示，请点此查看更多更全内容