Subscribe free to our newsletters via your
. Space Industry and Business News .




TECH SPACE
Storage system for 'big data' dramatically speeds access to information
by Helen Knight for MIT News
Boston MA (SPX) Feb 04, 2014


In the storage system, known as BlueDBM - or Blue Database Machine - each flash device is connected to a field-programmable gate array (FPGA) chip to create an individual node. The FPGAs are used not only to control the flash device, but are also capable of performing processing operations on the data itself.

As computers enter ever more areas of our daily lives, the amount of data they produce has grown enormously. But for this "big data" to be useful it must first be analyzed, meaning it needs to be stored in such a way that it can be accessed quickly when required.

Previously, any data that needed to be accessed in a hurry would be stored in a computer's main memory, or dynamic random access memory (DRAM) - but the size of the datasets now being produced makes this impossible.

So instead, information tends to be stored on multiple hard disks on a number of machines across an Ethernet network. However, this storage architecture considerably increases the time it takes to access the information, according to Sang-Woo Jun, a graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT.

"Storing data over a network is slow because there is a significant additional time delay in managing data access across multiple machines in both software and hardware," Jun says. "And if the data does not fit in DRAM, you have to go to secondary storage - hard disks, possibly connected over a network - which is very slow indeed."

Now Jun, fellow CSAIL graduate student Ming Liu, and Arvind, the Charles W. and Jennifer C. Johnson Professor of Electrical Engineering and Computer Science, have developed a storage system for big-data analytics that can dramatically speed up the time it takes to access information.

The system, which will be presented in February at the International Symposium on Field-Programmable Gate Arrays in Monterey, Calif., is based on a network of flash storage devices.

Flash storage systems perform better at tasks that involve finding random pieces of information from within a large dataset than other technologies. They can typically be randomly accessed in microseconds. This compares to the data "seek time" of hard disks, which is typically four to 12 milliseconds when accessing data from unpredictable locations on demand.

Flash systems also are nonvolatile, meaning they do not lose any of the information they hold if the computer is switched off.

In the storage system, known as BlueDBM - or Blue Database Machine - each flash device is connected to a field-programmable gate array (FPGA) chip to create an individual node. The FPGAs are used not only to control the flash device, but are also capable of performing processing operations on the data itself, Jun says.

"This means we can do some processing close to where the data is [being stored], so we don't always have to move all of the data to the machine to work on it," he says.

What's more, FPGA chips can be linked together using a high-performance serial network, which has a very low latency, or time delay, meaning information from any of the nodes can be accessed within a few nanoseconds. "So if we connect all of our machines using this network, it means any node can access data from any other node with very little performance degradation, [and] it will feel as if the remote data were sitting here locally," Jun says.

Using multiple nodes allows the team to get the same bandwidth and performance from their storage network as far more expensive machines, he adds.

The team has already built a four-node prototype network. However, this was built using 5-year-old parts, and as a result is quite slow.

So they are now building a much faster 16-node prototype network, in which each node will operate at 3 gigabytes per second. The network will have a capacity of 16 to 32 terabytes.

Using the new hardware, Liu is also building a database system designed for use in big-data analytics. The system will use the FPGA chips to perform computation on the data as it is accessed by the host computer, to speed up the process of analyzing the information, Liu says.

"If we're fast enough, if we add the right number of nodes to give us enough bandwidth, we can analyze high-volume scientific data at around 30 frames per second, allowing us to answer user queries at very low latencies, making the system seem real-time," he says. "That would give us an interactive database."

As an example of the type of information the system could be used on, the team has been working with data from a simulation of the universe generated by researchers at the University of Washington. The simulation contains data on all the particles in the universe, across different points in time.

"Scientists need to query this rather enormous dataset to track which particles are interacting with which other particles, but running those kind of queries is time-consuming," Jun says. "We hope to provide a real-time interface that scientists can use to look at the information more easily."

.


Related Links
Massachusetts Institute of Technology
Space Technology News - Applications and Research






Comment on this article via your Facebook, Yahoo, AOL, Hotmail login.

Share this article via these popular social media networks
del.icio.usdel.icio.us DiggDigg RedditReddit GoogleGoogle








TECH SPACE
Fujitsu returns to profit with healthy sales
Tokyo (AFP) Jan 30, 2014
Japan's Fujitsu swung back to profit in the three months to December thanks to brisk sales in PCs and networking services for public and business customers, as well as a weaker yen, the company said Thursday. The sprawling IT conglomerate said it logged a net profit of 12 billion yen ($117 million) in the third quarter of the fiscal year, against an 80.8 billion net loss a year earlier. ... read more


TECH SPACE
Oman orders NASAMS air defense system

A Proposal For The Space Debris Society

Storage system for 'big data' dramatically speeds access to information

Raytheon secures first international customer for its F-16 RACR AESA radar

TECH SPACE
MUOS Satellite Tests Show Extensive Reach In Polar Communications Capability

US Marines Reach Milestone For New General Dynamics-built Aviation CCS

Space squadron optimizes wideband communication constellations

GA-ASI and Northrop Showcase Unmanned Electronic Attack Capabilities

TECH SPACE
The go-ahead is given for Arianespace's February 6 flight with Ariane 5

SpaceX's next cargo mission to space station is Mar 16

Both payloads for Arianespace's next Ariane 5 flight are mated to the launcher

45th Space Wing Supports NASA Launch

TECH SPACE
Lockheed Martin Powers On Second GPS 3 Satellite In Production

India to launch three navigation satellites this year

NGC Wins Contract For GPS-Challenged Navigation and Geo-Registration Solution

20th Anniversary of Initial Operational Capability of the GPS Constellation

TECH SPACE
USAF Receives First B-1 Equipped with Boeing Integrated Battle Station

Launching the Fastest Plane of the Future

Canadian firm buys British, U.S. landing-gear manufacturing operations

USAF Orders Additional Boeing Combat Survivor Evader Locators

TECH SPACE
Integration brings quantum computer a step closer

New quantum dots herald a new era of electronics operating on a single-atom level

Dutch hi-tech group ASML profits dip despite record sales

2-proton bit controlled by a single copper atom

TECH SPACE
High resolution, digital bathymetry now available off-the-shelf

Savanna vegetation predictions best done by continent

Chinese scientists pinpoint source of Yangtze's main tributary

China to promote geological information industry

TECH SPACE
Asian ozone pollution in Hawaii is tied to climate variability

Cooperative SO2 and NOx aerosol formation in haze pollution

Made in China for us: Air pollution tied to exports

Delhi says air 'not as bad' as Beijing after smog scrutiny




The content herein, unless otherwise known to be public domain, are Copyright 1995-2014 - Space Media Network. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA Portal Reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. Advertising does not imply endorsement,agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. Privacy Statement