Business Unit Empowerment Through Declarative Systems

Business Unit Empowerment Through Declarative Systems Why IBM DB2 PureXML Rocks! Dan McCreary Dan McCreary & Associates September 11, 2007 M D Metadata Solutions DISCLAIMER The following views are my own and do not necessarily reflect the views of the Minnesota Department of Revenue or the State of Minnesota M

D 2 Presentation Abstract Many non-programmers feel that complex Java and .Net middle-tier system prevents business units from being able to quickly build and modify simple web applications (aka CRUD applications). DB2 version 9 has powerful support for quickly storing and searching complex XML documents. This feature combined with storing XML in a web browser using XForms makes middle-tier code unnecessary, putting non-programmers in control of applications. This presentation will cover a case study that allows a business unit to "draw pictures" to build simple web applications. These pictures (XML Schema diagrams) are transformed directly into XForms and stored in DB2 version 9. Rapid searching is provided using XQuery. Very little (if any) procedural code is involved in this process. The presentation finishes with a set of 10 recommendations that empower business units to

build and maintain their own applications and leverage the power of a central DB2 database. This presentation is appropriate for business and data strategists as well as DBAs and application architects. M D 3 Agenda How did we get here? Why trees not tables? Demos NIEM, XML Schemas, XForms Costs of translation from XML to OO to tables

XQuery in DB2 DB2 and your Enterprise Technology Strategy M D 4 Incoming! Has this web thing gone away yet? M X L

web M D 5 More New XML Data is Being Generated Than Relational Data Growing at 2X rate of total database market [IDC] XML is now pervasive in many of organizations Almost every sector has XML based standards "XML needs its own entry here to reflect its transformation of information XML is enabling the revolution of Web services and driving a database revolution." -- eWeek "25 More Top Technologies", September 18, 2006

M D 6 A Happy Partnership XForms DB2 PureXML M D 7 Strategic Consequences

Empowerment of business units to build and maintain their own applications with much less involvement of IT staff Dramatically reduced need for middleware IT staff using traditional procedural languages like Java and C# Evolution of DB2 to a web service data providor and core component of ESB M D 8 Did you Know That with XForms you can build rich-client web applications without writing a single line of JavaScript? That IBM DeveloperWorks has almost 300

articles that discuss XForms? That DB2 version 9 "PureXML" has fantastic support for native XML data types and supports W3C XQuery standards? M D 9 How did we get here? From punch cards to tables to trees to cubes and graphs M D

10 In the Beginning There were puchcards and there were 80 columns and the widths were fixed and batches of cards were called "decks and they were loaded into tables in core memory M D 11 And Codd and Date said Let there be relationships Person person_id

first_name last_name birth_date Role person_id project_id role_nm time_pct Project project_id project_nm description status and we had the relational databases and we had SQL

and we had joins and we had ER diagrams and we had PowerBuilder and all seemed right with the worlduntil M D 12 Sir Berners-Lee Said Let there be HTML Dans Homepage

Welcome!

Welcome to my web site!

About Me and we had URIs and we had HTTP and we had we links and resources and we had trees (not tables) and we had searchuntil M D 13 Customers Said.. We wanted to access our tableson the webin HTML! html

? head body h1 p a and there were objects and there was Java and JSP and there was ASP.Net and there were models, views, controllers and there was conversion from tables to objects to trees and trees to objects to tables and there were rooms and rooms of programmers M D

14 And there Was Chaos HTML Forms html Objects JSP/ASP head body h1 p Hibernate

a Serialization 100+ Frameworks OR Mapping JavaScript XML Struts JDBC SQL .Net AJAX M

D Sessions Stored Procedures And the number of languages you need to learn and the headcount of the IT department went upand up RDBMS 15 The Translation Pain Chain Name: Street: City: Zip:

Web Forms Objects RDBMS From web forms to objectsto SQL insertsto selectsto objects and back to web forms Many format translations M D 16 And the W3C Said

Let their be simplicitywhat if weallowed you to XML in the browserand database html head Database body model input Person label FirstName LastName Projects

Roles Project M D 17 A Single Line of SQL INSERT into MY_TABLE values(XMLPARSE(document ' ')) The equivalent SQL for the eCRV project would have required shredding the XML document into over 25 tables (i.e. 25 separate INSERT statements). M

D 18 Enter the Metadata Registry A metadata registry is a central location in the enterprise that the semantics or meaning of data elements is captured and maintained http://purl.org/dc/elements/1.1 ResourceTitleText The name given to the resource, usually by the Creator or Publisher.

Begin with the end in mindsemantic precision. M D 19 Demos NIEM Subset Generator http:// niem.gtri.gatech.edu/niemtools/ssgt Certificate of Real Estate Value http://proptax.mdor.state.mn.us:8088/exist/rest//db/crv/forms/submit/index.xhtml Contacts Demo

http://proptax.mdor.state.mn.us:8088/exist/rest/db/contacts/index.xhtml M D 20 M D 21 M D 22 XML Schema (XMLSpy Diagram)

Solid line means required Data Element Sequence of items Definition Dashed line means optional Cardinality Graphical notation that most non-programmers can learn in 20 minutes. M D

23 XForms User Interface (Generated) Use XSL to transform XML Schema directly into Xforms Use the metadata registry to extract element-specific rules Separate semantics from constraints M D 24 Metadata Driven Apps Data Flow NIEM MDR wantlist.xml

Local MDRs NIEM Sub Schemas .xsd import Document Constraint Schema .xsd xforms-global.css field-widths.css

patterns.xml read-only.xml screen-labels.xml import niem2xforms.xsl instance.xml import My-Form .xhtml This is a simplified version of the actual workflow. M D

25 How About Reporting on XML data? OK it is easy to get XML into DB2.. but how about reporting? How do I do joins on XML? How do I update XML documents? Can I mix SQL and XQuery? M D 26 DB2 Viper Architecture Many benefits of mix and match SQL/Xquery

Use the right tool for the right job M D 27 M D 28 M D 29 XQuery W3C Recommendation 23 January 2007

Created by experts from SQL industry IBM, Microsoft, Oracle IBM played a leadership role XQuery is to XML as the SELECT statement is to SQL M D 30 XQuery Structure FLWOR expression (pronounced flower) For select the sequence of nodes in the tree Let temporary variables and lists Where restrict results Order order results

Return return the data embedded in a tree M D 31 Sample XQuery for $d in db2-fn:xmlcolumn(depts.depts")//deptno let $e := db2-fn:xmlcolumn(emps.emps")//employee[deptno = $d] where count($e) >= 10 order by avg($e/salary) descending return { $d, {count($e)}, {avg($e/salary)} }

Returns a list of all big departments (over 10 employees), their headcount and average salary directly in XML format M D 32 XSLT and XQuery Functionally similar languages developed by the W3C Working together to ensure a common approach where appropriate. They share the same data model type system(XML Schema data types) function library

Both us XPath 2.0 as a sublanguage M D 33 What I like About XQuery XPath expressions are very short but can still easy to read (c.f. Perl) Consistent with XSL, XForms, XML Schema and Schematron (XML rules) You can extend XQuery using stored procedures Recursion simplifies many tasks M D 34

But Can I Do joins between XML documentsyes Extend XQueryyes Create functions with XQueryyes Integrate Java functionsyes M D 35 Joins in XQuery

declare variable $firstName external; { for $v in db2-fn:xmlcolumn(videos)//video, $a in db2-fn:xmlcolumn(actors)//actors/actor where ends-with($a, $firstName) and $v/actorRef = $a/@id order by $v/year return List of videos in one file Return a list of all videos } That have an actor with a

given first name. M D 36 XQuery Update Adds XQuery expressions to: Insert Update Delete XML documents Still in W3C Working Draft Last call for Comments is Oct. 2007 Should be full recommendation by 2008

Available in DB2 9.1 as a stored procedure Additional support in DB2 9.5 M D 37 Stored Procedures For Update CREATE PROCEDURE ECRVADMIN.INCREMENT_COUNTY_CRVID (IN COUNTY VARCHAR(255), OUT SUCESS INT) LANGUAGE SQL BEGIN SET ARG1 = CONCAT( ') CPMCALL DB2XMLFUNCTIONS.XMLUPDATE( ARG1, select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?);

CALL DB2XMLFUNCTIONS.XMLUPDATE( ' ? +1 ', 'select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?); CALL DB2XMLFUNCTIONS.XMLUPDATE( ' ?+1 ', 'select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?); M END D 38

Sidebar: The Semantic Web Subject Node Predicate Object Node Node Node Node Node M

Note that the semantic web infrastructure is based on graphs, not trees Each node in the graph can be a URL External web sites can be used to create joins based on a URL D 39 Strategic Impact Consider this: 50% of Applications are simple CRUD Applications Create, Read, Update, Delete Complex business logic can be stored in an XML-based rules engine such as Schematron How will this transform an organization? How will it allow business units to build and

maintain IT applications? How will it allow the business to be responsive to custom needs? M D 40 Change Where the Line is Drawn Requirements Requirements BAs SME Developers

vs. Graphical Requirements and Specifications SME/BA M IT Staff Shorten the distance between the business unit and the IT staff D 41 Many Small vs. One Large XForms

XQuery XPath XSLT XML Schema Requirements Schematron It is easier to build a GUI on a small, precise language than a single large language M D 42

Data Stewardship/Governance Importance of the business units role in defining data elements semantics on-thewire How to write and maintain enterprise definitions Concise Precise Distinct Non-circular Unencumbered M D 43 Security and Performance

Security and access control based on URL patterns A middle tier still needed to cache web pages that have similar URLs Deep REST considerations (dont regnerate a pages if you dont have to) Stateless web server farms can still be used M D 44 Modules and Quality Assurance XQuery Modules allow encapsulation of functions to manipulate complex XML Need better unit testing tools for XQuery (similar to JUnit)

Need support for regression testing frameworks (similar to CruiseControl) M D 45 Parker Projection Relative Code Base 100% Proce dural c

ode (J e ativ r a l Dec ava, J co avaS c ML T

H X de ( ript, V B ,X S S ,C , C# , C SL ++)

) rms o F ,X y r e Qu T, X Time M D Source: Jason Parker, Minnesota Department of Revenue, November 2006

46 10 Business Empowerment Steps 1. 2. M Make data stewardship a business unit imperative Start by putting a simple glossary of terms and their definitions in a database 3. Build a simple metadata registry based on a simple taxonomy 4. Build a metadata "shopping cart" 5. Build exchange documents using XML Schemas that import your metadata shopping lists 6. Transform XML Schemas into XForms 7. Put your XML data in DB2 XML data types (no

shredding!) 8. Use XQuery for reports directly into XML and HTML 9. Express level-two business rules by selecting XPath expressions 10. Build XML-centric ESB with DB2 databases as sources D 47 Next Steps 1. Proof of Technology (POT) hands on labs 2. Get the evaluation versionstart with some small XForms and XQueries 3. Create a pilot project goal small middle tier 1. Pick a "friendly" business unit 2. Find a medium sized problem 4. Train staff with XPath-centric tools

5. Roll out to the enterprise in stages M D 48 M D 49 M D 50 Developerworks Article

http://www.ibm.com/developerworks/library/x-xformsniem M D 51 XForms Wikibook http:// en.wikibooks.org/wiki /Xforms Over 75 working XForms example programs M D 52

XQuery Book XQuery Priscilla Walmsley Tutorial and Reference OReilly - 2007 Everything except XQuery update M D

53 Devx Article Semantics and the Evolution of Specialized Languages http://www.devx.com/semantic/Article/34805 M D 54 The IBM Workplace XForms Video http://www.ibm.com/developerworks/library/x-xformspointcl ick.html One-click generation of XML instance data Drag-and-drop of complex type elements to create

groups of UI controls that are already bound to data Drag-and-drop of UI controls from the design palette Drag-and-drop association of data to UI controls created from the design palette Wizard-based generation of data-driven hiding/showing of groups of UI controls Wizard-based generation of tables, including add row and delete row buttons Wizard-based generation of intelligent row calculations and column summations M D 55 More Resources

Keywords: XForms, XQuery, PureXML This presentation: http://www.danmccreary.com/presentations/db2ug M D 56 Useful Links DB2 9 Trial http://www.ibm.com/software/data/db2/v9 DB2 9 Express-C - Free limited cpu version http://www.ibm.com/software/data/db2/udb/db2expres s/download.html IBM DeveloperWorks http://www.ibm.com/developerworks/xml

http://www.ibm.com/developerworks/db2 IBM DB2 9 PureXML Wiki http://www.ibm.com/developerworks/wikis/display/db2 xml/Home M D 57 Thank You! Dan McCreary Dan McCreary & Associates Enterprise Data Architecture and Strategy Consulting [email protected] http://www.danmccreary.com Cell: (612) 986-1552 Home: (952) 931-9198

M D 58

Recently Viewed Presentations

  • Chapter 3 Fundamental Differences Between Goods and Services

    Chapter 3 Fundamental Differences Between Goods and Services

    Use physical evidence/tangible clues. The physical characteristics that surround a service to assist consumers in making service evaluations, such as the quality of furnishings, the appearance of personnel, or the quality of paper stock used to produce the firm's brochure
  • Comp 311 Lecture 1 Course Overview and Culture

    Comp 311 Lecture 1 Course Overview and Culture

    Top Down Parsing What is a context-free grammar (CFG)? A recursive definition of a set of strings; it is identical in format to the data definitions used in Comp 210 except for the fact that it defines sets of strings...
  • Presentación de PowerPoint - WordPress.com

    Presentación de PowerPoint - WordPress.com

    funciÓ persuasiva. en aquest tipus de comunicaciÓ el missatge estÀ centrat en que el receptor quedi seduÏt i atret pel contingut del missatge de manera que actuÍ segons es propossa l'emissor.És el cas de la publicitat comercial, les campanyes de...
  • Re-Envisioning High School CS and Getting it Taught

    Re-Envisioning High School CS and Getting it Taught

    In progress: Framework: First two levels (Big ideas & Key concepts) Still to come: Framework: Third level (Enduring understandings) Curriculum/Materials Test AP GSC's Big Ideas 1. Computing is a creative activity that draws on a wide variety of fields, such...
  • Chapter 1 Structure and Bonding - Milwaukie High School

    Chapter 1 Structure and Bonding - Milwaukie High School

    A buffer solution is a solution of:. A weak acid or a weak base . and. The salt of the weak acid or weak base. Both must be present! A buffer solution has the ability to resist changes in pH...
  • SESSION CODE: SVR316 Brett Chase Technical Director APJ

    SESSION CODE: SVR316 Brett Chase Technical Director APJ

    The MVA helps improve your IT skill set and advance your career with a free, easy to access training portal that allows you to learn at your own pace, focusing on Microsoft technologies. What Do I . get for enrolment?...
  • The National Archives' Records Management Standard requires:

    The National Archives' Records Management Standard requires:

    At least 50% questions in Caldicott are linked with ISMS compliance - some examples below. ISMS states a Caldicott Guardian/Lead should be appointed - Link QG1. ISMS states a Business Continuity plan should be in place and tested - Link...
  • U.S. General Services Administration Reporting Excess Personal Property

    U.S. General Services Administration Reporting Excess Personal Property

    DOI/AMD is limited to FSG15 aircraft and parts only, Dept. of Treasury IRS Vehicle Sales is limited to passenger vehicles, an approved waiver must be obtained through GSA OGP and property is then not required to be sold via a...