SYS-CON Media
 Register Now!
Save $200
Register before October 17th to SAVE! ... and also receive a FREE copy of the Best-Selling AJAX Book, a $119 Value!
Untitled Document
2008 East
Platinum Sponsor
Untitled Document
2008 East Gold Sponsors
Untitled Document
2008 East Exhibitors
Untitled Document
2008 East Media Sponsors
Untitled Document
2008 Association Sponsor
SYS-CON Media
2007 West Sponsors
Goingtomeet.com Conference Directory
SYS-CON Media
2007 East Sponsors
Untitled Document
2008 SYS-CON Events

Can't Miss RSS Feed
Subscribe to the AJAXWorld.com RSS Feed & Get All The Conference News As It Happens!

2008: Decision Year for RIAs - October 20-22, 2008 San Jose


Real-time web data services in action at German stock exchange
Real-time web data services in action

As the culmination of a four-part series on web data services (WDS), we examine a fascinating use-case for data services with Deutsche Börse Group in Frankfurt, Germany. An innovative information service recently created there highlights how real-time content and data assembled from various online sources scattered across the Web provides a valuable analysis service.

The offering supports energy traders seeking to track global fluctuations and micro trends in oil and other related markets. But, the need for real-time and precise data affects more than energy traders and financial professionals. More than ever, all sorts of businesses need to know what's going on in and what's being said about their respective markets, products, and services.

In this series with Kapow Technologies, we've examined the need for WDS and ways that WDS and related tools can be used broadly to solve these problems. Now, we are going to learn the full story of how Deutsche Börse took web data resources, and not only efficiently assembled knowledge from automated robots, cleansing tools, and analytics management, but from these capabilities they also created high value and focused WDS offerings onto itself.

Thanks for joining us, as we take an in-depth look at how the market for WDS has shaped up and then hear directly from the leader of the Deutsche Börse project, as well as from a key supplier that supported them in accomplishing their web services goal.

Access the full series of podcasts on web data services:

So, to learn more about WDS as a business, please welcome our guests, Mario Schultz, director of Energy Facts at Deutsche Börse Group, and Stefan Andreasen, CTO at Kapow Technologies. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Gardner: It's interesting to me that we've moved beyond a level of static information to dynamic information and yet we still haven’t taken full advantage of everything that’s being developed and created across the Web.

But today’s market turbulence demands that we do that. We have to move into an era where we can take quality data and provide agility into how we can consume and distribute it. We're dealing with more diverse data sources. That means we need to have completeness and we need to be comprehensive, in order to accomplish the business information challenges each business faces.

The need now is for flexible, agile, and mixed sourcing of services and data together.



The need now is for flexible, agile, and mixed sourcing of services and data together. The content is often portable. That means it's ubiquitous across mobile devices and social networks in such a way that real-time analytics becomes extremely important.

The use of data as a business is now coming to the fore. We're beginning to see value, not from just the assimilation of data for use internally, but as more and more businesses are starting to take advantage of the data that they create and have access to. They share that with their partners, create ecosystems of value, and then even perhaps sell outright the information, as well as insights and analysis from that information.

Schultz: Deutsche Börse is the German stock exchange in Frankfurt, Germany, and we offer all kinds of products and services around on-exchange trading and the adjacent processes. For several years now, I've been responsible for developing new products and services around information for on-exchange or off-exchange trading. This is why we've invented and developed the Energy Facts service.

We developed new products and services where we could transform our know-how and this real-time connection, aggregation, and dissemination of data to other business lines. This is why we looked into the energy trading sector, mainly focused on the power trading here in Europe.

I began by working on the exchange of information that we have in our own systems. We were proceeding with our ideas of enhancing our services and designing new products and services. We were then looking into the Web and trying to get more information from the data that we gather from websites -- or somewhere else on the global Web -- and to integrate this with our own company's internal information.

Everything we do focuses on the real-time aspect. Our use of web data services is always focusing on the real-time aspects of this.

At Deutsche Börse, we have something that’s called Xetra, our electronic trading system for cash products. We have Eurex, our derivative business line, which is worldwide, well-known, where you can trade other derivatives on that platform.

We have a main system called CEF. It is our backbone IT solution for delivering data in real-time with milliseconds optimization. The data is mainly coming from our internal IT systems, like Xetra and Eurex, and we deliver this data to the outside world.


In addition, we calculate all the relevant indices, like the DAX, the flagship index for the German markets with 30 instruments, and more than 2,000 -- or nearly 3,000 -- indices that are distributed over the well-known data vendors, for example, Bloomberg or Reuters. They are our main distribution networks, where we are delivering all our information.

Germany is currently the most important market for energy and power trading in the middle of Europe.



By talking to well-known players in the market, we quickly recognized that we could build up a very powerful and fundamental data models. You have to collect all the relevant information to get an overview and to get an estimate about the price, in this case, where power could develop and in which direction it could develop.

Traders are looking into the fundamental factors that affect the price of the energy or the power that you trade, whether it’s oil or whatever. That’s how we started with power trading. You have the wind and other weather factors. You have temperature. You have the availability of power plants. So, you try to categorize and summarize these sectors. It's called the supply and the demand side regarding this energy trading.

Fundamental data models

The main issue and main task in the beginning was to collect the relevant data. Quite quickly, we were able to set up a big list of all relevant data sets or sources, especially for Germany and some adjacent countries. We came up with something around 70, 80, or even 100 different sources on the Web to grab information from. So, the main issue was how to collect and grab all this data in a manageable way into one data base. That was the first step.

In the second step, Kapow came into this play. We recognized that it’s really important to have a one-stop shopping inbound channel that collects all the information from these sources, so that you don’t have to have have several IT systems, or your own program, JavaScript, or whatever to get the information.

I wanted to have a responsible product manager for this project or for this new product. From the beginning, I had to have a good technology in place that would be able to handle all these kind of sources from the Web.

We recognized that there are so many different data formats that we had to grab. There are all these different providers of information in Germany and other European countries. They have their own websites. Some give the data in HTML format. Others use XLS, CSV, or even PDFs.

Kapow tells us how to get this information from these different sources in quite different formats. This is a manageable way, with a process-driven or graphical user interface (GUI) driven tool, that would use the effort, the personal, the manpower efforts to collect and grab the data.

Not only websites

Currently, we have 70 or 80 sources that we're grabbing. It's not only websites, but we have some third-party providers that are delivering information, for example, weather, temperature, and things like that. We have providers giving data via FTP service, and we even use Kapow for grabbing data from these third-party players. As I said, it's a one-stop shopping solution to get everything via one channel.

The value-add was to grab all this data into one common data format, one database, so we would be able to deliver this data to the vendors via web tool, web terminal, or even our existing CEF data feeds. A lot of the players in the market are trying to collect this data by themselves, or even manually, to get an overview of where the power price would develop over the next day, hours, weeks, months, whatever.


Andreasen: This is an extremely impressive service that Mario just showed us here, and I'm sure, if you're dealing with buying and selling energy, this is a must for you to be sure you made the right decision.

If these data sources exist somewhere on the Web, we can actually grab them where they are. What you traditionally do with information gathering is that you call every company or every entity that has data and ask them, "Will you please provide the data in this or this format?" But, with Kapow Web Data Services, you can just grab the data, wherever it is on the Web, and assemble this valuable data source much easier and much faster.

Businesses are relying more and more on data to make the right decision, and their focus is on quality, completeness, and agility. Let's be more practical here and ask how you actually get this data.

There is a term, data integration, which is about accessing the data and providing it in standard API, so that you can actually leverage the measure of business application.

Energy Facts is accessing this data at the 70-80 different data sources, as Mario said, and providing it as a feed that depends on the volatility of the different data sources. Some of the data delivers every minute, and some deliver every four hours, etc., based on how quickly the data source changes. WDS is all about getting access to this data where it resides.

There are really two different kinds of data sources. One set of data sources is more like a real-time source data source. Let's say you go to a patent directory, and there are probably millions of patents. In that case you would use Kapow Data Server to wrap that data source into a service layer, and then you would be able to do real-time, as soon as you get real-time results back. So, that's real-time access, where you have vast amount of information.

Actually, all styles exist, but there is a tendency for many companies to actually access the data where it is, rather than trying to consolidate it to a new place.



The other scenario, and I think that's more what we see in the Energy Facts example here, is where you have a more limited data source, and you are actually trying to do a consolidation of the data into a database, and then you use that database to serve different customers or different applications.

With Kapow, you can actually go in and access the data, if you can see them on your browser. That's one thing. The other thing you need to do to make this data available to your business application is to transform and enrich the data, so that it actually matches the format that you want.

For example, on the website, it might have the date saying, "2 hours ago" or "3 minutes ago" and so on. That's really not useful. What you really want is a time stamp with the hour, the second, the minute, the months, the day, the year, so you can actually start comparing these. So, data cleansing is an extremely important part of data extraction and access.

The last thing, of course, is serving the data in the format you need. That can be a database, if you're doing consolidation, or it can be as an API, if you are doing more of a federated access to data, and leaving the data where it is.

Actually, all styles exist, but there is a tendency for many companies to actually access the data where it is, rather than trying to consolidate it to a new place.

More examples on data as a service

Go to our website and download a white paper from one of our customers, called Fiserv. It's a large financial services company in the U.S. Fiserv has a lot of business partners, actually they have more than 300 banks in more than 10 countries as business partners. Because they're selling services, it's incredibly important for them to also monitor their customers to understand what's happening.

They had lot of people who logged into these 300 partner banks every day and grabbed some financial information, such as interest rates, etc., into an Excel spreadsheet, put it into a database, and then got it up on a dashboard.

The thing about this is that, first, you have a lot of human labor, which can cause human errors, and so on. You can only do it once a day, and it's a tedious process. So what they did is got Kapow in and automated the extraction of this data from all their business partners -- 300 banks in more than 10 countries.

They can now get that data in near real-time, so they don’t have to wait for data. They don’t have to go without on the weekend, because people are not working. They get that very business critical insights to the market and their partners instantly through our product.


Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download the transcript. Learn more. Sponsor: Kapow Technologies.

You may also be interested in:

Latest AJAXWorld RIA Stories
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our ...
Technological progress can be expressed as layers of abstraction - higher layers are built on top of lower layers treating them as abstract black boxes with known interfaces. A serverless approach represents an inflection point that entirely separates the runtime layer from the u...
Your job is mostly boring. Many of the IT operations tasks you perform on a day-to-day basis are repetitive and dull. Utilizing automation can improve your work life, automating away the drudgery and embracing the passion for technology that got you started in the first place. In...
Big Switch's mission is to disrupt the status quo of networking with order of magnitude improvements in network e ciency, intelligence and agility by delivering Next-Generation Data Center Networking. We enable data center transformation and accelerate business velocity by delive...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and l...
Untitled Document

Call 201 802-3020 or Click Here to Save $200!

Register Today and
Save $200

Your registrations includes: Golden Pass Delegates will receive full conference access on October 20-22, 2008 including: Lunch and Coffee Breaks, Collectible Bag and Archives of all sessions on DVD. Includes access to all Conference Sessions including the Technical Sessions, Exhibits, Keynotes, Vendor Technology Presentations, and Power Panels.


Sponsorship Opportunities

AJAXWorld offers the undisputed best platform to position your company as a leading vendor in the fast-emerging marketplace for AJAX and Enterprise Web 2.0.


Please call
(201)802-3020


Who Should Attend?

 CTOs & VPs of Engineering
 Directors of Technology
 Sr. User Interface Architects
 Front-End Engineers
 VCs & Industry Analysts
 Directors of Business Development
 Software Engineers
 Senior Architects
 Application Programmers & Software Developers
 Project Managers
 Web Programmers & Designers
 Companies & Organizations that need to stay in
  front of the latest Web technology

AJAXWorld Security Bootcamp

Introducing at AJAXWorld RIA Conference 2008 West the world's first-ever full, one-day immersive "AJAX Security Bootcamp" - led by one of the world's foremost AJAX security experts and teachers, HP's Billy Hoffman.

View the full one-day schedule



AJAXWorld 2008 West - Tracks

Track 01: Enterprise RIAs
Track 02: Frameworks & Toolkits
Track 03: Web 2.0 & Mashups
Track 04: Hot Topics
Track 05: The Future of the Web
Track 06: iPhone Developer Summit



Brought To You By:

AJAXWorld Magazine is the pre-eminent independent vendor-neutral resource for the fastest growing new segment of the software business: entirely Web-based applications and experiences.

Download the Latest Issue!

AJAXWorld Webcasts



SYS-CON EVENTS


AJAXWorld Keynotes & Power Panels

2008 SYS-CON TV Keynotes: Can We Fix the Web? By Douglas Crockford - by Douglas Crockford
2008 SYS-CON TV Keynotes: 2008: The Year of the RIA - by Anthony Franco
2008 SYS-CON TV Power Panel: The Business Value of RIAs
2008 SYS-CON TV Power Panel: What Lies Beyond AJAX
2007 SYS-CON TV Keynotes: Why Web 2.0 for the Enterprise Is Far More Than Just a Facelift - by Ted Farrell
2007 SYS-CON TV Keynotes: Fueling the Next Generation Web: A Peek Behind the Green Curtain - by Bob Brewin
2007 SYS-CON TV Keynotes: AJAX in the Balance - by Joe Stagner

AJAXWorld Sessions on SYS-CON.TV

· Bill Scott - Yahoo! UI Library
· David Heinemeier Hansson - AJAX on Rails
· Jesse James Garrett - Elements of User Experience
· Dion Hinchcliffe - Real World AJAX
· Eric Miraglia - Open Source AJAX Development
· Paul Rademacher - Mashing Up Your Web Application
· Adam Sah - Google Gadgets
· Doug Crockford - An Introduction to JavaScript
· David Linthicum - Enterprise Web 2.0
· Patrick Grady - The Imagination & Experience Web

AJAXWorld...All The AJAX Rock Stars in One Spot!


Past Events Archive

SOAWorld Conference & Expo 2008 East
soa2008east.sys-con.com
Virtualization Conference & Expo 2008 East
virt2008east.sys-con.com
AJAXWorld 2008 Conference & Expo East
ajaxmar08.sys-con.com
SOAWorld Conference & Expo 2007 West
www.soaworld2007.com
Virtualization Conference & Expo 2007 West
virt2007west.sys-con.com
AJAXWorld 2007 Conference & Expo West
ajaxoct07.sys-con.com
SOAWorld Conference & Expo 2007 East
soa2007east.sys-con.com
Virtualization Conference & Expo 2007 East
virt2007east.sys-con.com
AJAXWorld 2007 Conference & Expo East
ajaxmarch07.sys-con.com
Other SYS-CON Events
events.sys-con.com

Join Over 10,000 Early AJAX Adopters
Who Have Attended AJAXWorld
• A&R Edelman
• Academic Enterprise
• Accoona Corp [2 delegates]
• Acxiom
• Adams Capital Management
• Adaptive Edge
• Adaptive Path
• Adobe Systems Incorporated [21 delegates]
• Adobe Systems Romania
• Ajax13
• All Risks, Ltd.
• alliance
• Alliance For Community Care
• AlphaDetail Inc
• Altera Corporation
• Amazon.com [6 delegates]
• Appeon Corporation [2 delegates]
• Apple Computer [5 delegates]
• Apress [3 delegates]
• Arkivio
• ASA
• Astute Solutions
• Avaya Inc [2 delegates]
• Avenda Systems
• Avenue A | Razorfish [3 delegates]
• Axcella, LLC [2 delegates]
• Aximsoft
• Azimyth
• Backbase USA Inc. [4 delegates]
• BAE Systems [2 delegates]
• Bank of America [2 delegates]
• Barkley Evergreen & Partners Interactive
• Bayview Financial [2 delegates]
• BEA Systems [3 delegates]
• Billeo
• BMC Software, Inc. [2 delegates]
• Borland Software Corporation
• Bradford Technologies, Inc [2 delegates]
• Brilliance
• Brocade Communications Systems, Inc. [2 delegates]
• Brookside Capital LLC
• Brulant
• Bungee Labs, Inc [6 delegates]
• Bureau of Labor Statistics
• BUZ Interactive
• Cadena Software
• Calix Networks
• Callidus Software [2 delegates]
• Cambia Security
• Carnegie Mellon West
• Cautella, Inc.
• CBSA
• Celequest [3 delegates]
• Change Vision, Inc.
• Charles E. Kenney, CPA
• Charles Schwab & Co., Inc. [8 delegates]

   read more...


ADVERTISE   |   MAGAZINE SUBSCRIPTIONS   |   FREE BREAKING-NEWSLETTERS!   |   SYS-CON.TV   |   BLOG-N-PLAY!   |   WEBCAST   |   EDUCATION   |   RESEARCH

.NET Developer's Journal - .NETDJ   |   ColdFusion Developer's Journal - CFDJ   |   Eclipse Developer's Journal - EDJ   |   Enterprise Open Source Magazine - EOS
Open Web Developer's Journal - OPEN WEB   |   iPhone Developer's Journal - iPHONE   |   Virtualization - Virtualization   |   Java Developer's Journal - JDJ   |   Linux.SYS-CON.com
PowerBuilder Developer's Journal - PBDJ   |   SEO / SEM Journal - SJ   |   SOAWorld Magazine - SOAWM   |   IT Solutions Guide - ITSG   |   Symbian Developer's Journal - SDJ
WebLogic Developer's Journal - WLDJ   |   WebSphere Journal - WJ   |   Wireless Business & Technology - WBT   |   XML-Journal - XMLJ   |   Internet Video - iTV
Flex Developer's Journal - Flex   |   AJAXWorld Magazine - AWM   |   Silverlight Developer's Journal - SLDJ   |   PHP.SYS-CON.com   |   Web 2.0 Journal - WEB2

SYS-CON MEDIA:   ABOUT US   |   CONTACT US   |   COMPANY NEWS   |   CAREERS   |   SITE MAP
SYS-CON EVENTS  |  AJAXWorld Conference & Expo  |  iPhone Developer Summit  |  OpenWeb Developer Summit  |  SOA World Conference & Expo  |  Virtualization Conference & Expo
INTERNATIONAL SITES:   India  |  U.K.  |  Canada  |  Germany  |  France  |  Australia  |  Italy  |  Spain  |  Netherlands  |  Brazil  |  Belgium
 Terms of Use & Our Privacy Statement     About Newsfeeds / Video Feeds
Copyright ©1994-2008 SYS-CON Publications, Inc. All Rights Reserved. All marks are trademarks of SYS-CON Media.
Reproduction in whole or in part in any form or medium without express written permission of SYS-CON Publications, Inc. is prohibited.