How IBM volunteers propose to solve IRRI’s data problem

MANILA, Philippines - Even for research organizations like the International Rice Research Institue (IRRI), the premier research organization dedicated to reducing poverty and hunger through rice science, big data is a complex problem.

As data sets that organizations handle get bigger and more complex, traditional data processing applications and computing power do not always keep up.

Over the years, IRRI has acquired hardware that can store data up to 200 TB and has solicited public storage of data for free from Amazon Web Services (AWS). But it isn’t enough or sustainable. A cost-effective, high-level IT architecture to meet the institutions’ data demands is needed.

A three-man team from the IBM Corporate Service Corps has worked with the organization for one month this year to sort out its big data problem. The result is a technology platform, as well as a capacity planning process, that could meet researchers’ current and future demand for computing power.

“Research is being negatively impacted by limited resources and limited access to resources in computing power,” says Michelle Kennedy, one of the members of the IBM-IRRI Big Data Team.

The team found out during the pro-bono consulting work that the Advanced Science and Technology Institute (ASTI), being funded by the Philippine government to support the scientific community, has available machines and compute power for science and research organizations like IRRI at a very low cost or even for free.

ASTI’s mandate include providing support to and conducting scientific research and development in the advanced field of information and Communication Technology (ICT ) and microelectronics, as well as undertaking long-term researches to strengthen and modernize the science and technology infrastructure.

 Few, however, are taking advantage of this government resource.

Richard Chang, another member of the IBM-IRRI Big Data Team, says there were two barriers. First, there was the notion held by researchers that data connection is too slow for data transfers between the organization’s base and ASTI and not often fast enough for research needs. And second, there is always that uncertainty about government support as projects can always get defunded anytime leaving the organizations or users hanging.

To dispel the notion about inadequate connectivity, Chang says they worked with a team of IT experts to test and optimize the connection speed at IRRI and the data transfer rates between IRRI and ASTI and between IRRI and the cloud.

The test results show that the connection is actually good and the team was still able to increase speed 10 times using one of its servers, uploading at 500 Mbps per second at more than 5TB in one day.

Moreover, the ‘one size fits’ all approach to connectivity is not always effective. So the team introduced a tiered model approach to connectivity – from the most expensive to the least expensive forms of connectivity depending on need. The IT architecture was also diversified – on premise at IRRI, in ASTI and the cloud.

“Big data is not only a technology problem,” says Roberto Roffo, another member of the IBM-IRRI Big Data Team. “Partnership is a crucial component of the solution.”

The team brokered a formal collaboration agreement with ASTI for two years for the use of computing resources. “That kind of clarity is good for both parties as it provides peace of mind and helps capacity planning,” Chang says.

V. Bruce J. Tolentino, deputy director general for Communication and Partnerships, IRRI.

The agreement includes the 200+ cores committed by ASTI to IRRI’s C4 Rice project for high-resolution phenotyping for a minimum of two years. Before the end of the year, six more agreements will be signed that will expand the partnership.

V. Bruce Tolentino, deputy director general for communication and Partnerships, IRRI, says these are institutional level agreements between the IRRI and ASTI; and IRRI and Department of Science and Technology (DOST).

“That means we can bring all of the agreements in a common framework and it is within the overview of the agreements that IRRI has with the government of the Philippines,” he says.

To ensure that resources are being used effectively, a quarterly report on active IRRI-ASTI collaboration will be released. This is also expected to help increase communication between IRRI scientists and ASTI.

Pro-bono consulting work

Fifteen IBM employees (including the three who worked on IRRI’s big data problem) from 10 different countries were chosen this year to render pro-bono consulting work as part of the IBM Corporate Services Corps in the Philippines.

Under the program, IBM employees spend four weeks in groups of 10 to 15 working collaboratively with their host government and community counterparts to develop blueprints that address issues including economic development, energy, and transportation, education and healthcare.

The recipient of this volunteer work this year is the IRRI and the Southeast Asian Regional Center for Graduate Study and Research in Agriculture (SEARCA), both based in Los Banos, Laguna.

David Raper, lead, Asia-Pacific and China, IBM’s Corporate Citizenship, says IBM places a great deal of importance on finding the right partners so that the work they do can be implemented and is sustainable. “The model of the IBM Corporate Service Corps is designed to come up with recommendations that make sense to that organization,” he says.

Thus, the choice of the organizations every year depends on factors like the capability of the organization to successfully host the IBM team and also the capability to implement the recommendations that the team will suggest, as well as issues of growth and development such as agriculture, education and healthcare.

Aside from the Big Data team, there were four other teams that worked on projects for both organizations, including the improvement of the IRRI Library, the enhancement of the web and country briefs, expanding training capacity of the IRRI Science Academy and support for the SEARCA Knowledge Management.

“We really believe that with these projects, we are bringing IBM’s expertise and capacity to organizations. If we are to put a consulting value to the project, it will be in the order of $400,000, but what is important is that we are able to unlock much more value for the organizations that we are working with,” says Raper.

The 15 were chosen by a rigorous process that screen applicants from all over the world.

“We also chose this,” says Chang. “We chose to work on this mission, to leave our families behind because we want to give back, we want to use the skills that we acquired in the public and private sector and to contribute to the missions we heard so much here. I guess we are also committed to social responsibility.”

By end 2015, IBM CSC has already dispatched approximately 2,800 IBM employees originating from over 60 countries on engagements to 38 countries for consulting work. Photos by Jojo Mamangun