. Space Industry and Business News .

Tool enables scientists to uncover patterns in vast data sets
by Staff Writers
Washington DC (SPX) Dec 20, 2011

What might we be missing in large datasets? If researchers printed on paper each potential relationship in a recent data set containing abundance levels of bacteria in the human gut, the stack of paper would reach to a height of 1.4 miles, 6 times the height of the Empire State Building. Credit: Sigrid Knemeyer.

With support from the National Science Foundation, researchers from the Broad Institute and Harvard University recently developed a tool that can uncover patterns in large data sets in a way that no other software program can.

Called Maximal Information Coefficient or MIC, the tool can can tease out multiple, recurring events or sets of data hidden in health information from around the globe, or in the changing bacterial landscape of the gut or even in statistics amassed from a season of competitive sports--and much more.

Part of a suite of statistical tools called MINE for Maximal Information-based Nonparametric Exploration, MIC has the ability to sort through today's mass of research variables--from attempts to track hurricanes, efforts to model earthquakes, endeavors to identify the Higgs Boson and efforts to glean insights from affecting the world economy and social networking interaction.

Researchers currently use advanced technology to gather big, complex, data sets, which may be incredibly useful in enhancing system understanding, if, in fact, vast amounts of data can be organized so that telling information may be extracted.

Sophisticated computer programs research these data sets with great speed, but fall short in even-handedly detecting different kinds of patterns in large data collections, essential for more sophisticated analysis.

One of the greatest strengths of this newly discovered tool within MINE is its ability to detect and analyze a broad spectrum of patterns and characterize them according to a number of different parameters a researcher might be interested in.

Other statistical tools work well for searching for a specific pattern in a large data set, but cannot score and compare different kinds of possible relationships. Researchers can also use MINE to generate new ideas and connections.

The researchers report their findings in the Dec. 16th issue of the journal Science.

Related Links
National Science Foundation
Space Technology News - Applications and Research

Get Our Free Newsletters Via Email
Buy Advertising Editorial Enquiries


. Comment on this article via your Facebook, Yahoo, AOL, Hotmail login.

Share this article via these popular social media networks
del.icio.usdel.icio.us DiggDigg RedditReddit GoogleGoogle

Supercomputers Take a Cue From Microwave Ovens
Berkeley CA (SPX) Dec 06, 2011
As sophisticated as modern climate models are, one critical component continues to elude their precision-clouds. Simulating these fluffy puffs of water vapor is so computationally complex that even today's most powerful supercomputers, working at quadrillions of calculations per second, cannot accurately model them. "Clouds modulate the climate. They reflect some sunlight back into space, ... read more

Research could improve laser-manufacturing technique

German company finds rare earths resources in Magadascar

Apple scores hit on HTC in US patent case

Can science predict a hit song

Satellite Tracking Specialist, Track24, wins Canadian Government Contract

Airman brings space to ground forces

Astrium achieves Initial System Acceptance on Yahsat programme

Northrop Grumman Awarded Microscale Power Conversion Contract

Vega moves closer to its first liftoff

Arianespace Signs First launch contracts for Vega

Arianespace Completes 2011 Launch Manifest With Successful Soyuz Campaign

Soyuz is cleared for its second Arianespace launch from the Spaceport

Lockheed Martin Delivers GPS 3 Pathfinder Satellite to Denver on Schedule

Galileo in tune as first navigation signal transmitted to Earth

Glonass satnav system targets Latin America and India

Lightweight GPS tags help research track animals of all sizes

Qantas reaches agreement with engineers

Removing sulfur from jet fuel cools climate

EU unyielding on airline carbon rules despite US pressure

Cathay announces economy class upgrade

Quantum Computing Has Applications in Magnetic Imaging

Sharpening the lines could lead to even smaller features and faster microchips

Optical Fiber Innovation Could Make Future Optical Computers a 'SNAP'

New method for enhancing thermal conductivity could cool computer chips, lasers and other devices

SMOS detects freezing soil as winter takes grip

NASA Gears Up for Airborne Study of Earth's Radiation Balance

Study Shows More Shrubbery in a Warming World

Astrium awarded Sentinel 5 Precursor contract

Beijing hits 'blue sky' target despite bad air

Mercury releases into the atmosphere from ancient to modern times

Keeping our beaches safe

Christmas shopping hampered as Milan battles smog


The content herein, unless otherwise known to be public domain, are Copyright 1995-2012 - Space Media Network. AFP and UPI Wire Stories are copyright Agence France-Presse and United Press International. ESA Portal Reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. Advertising does not imply endorsement,agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. Privacy Statement