For the past decade, the National Weather Service has been plagued by failures in disseminating critical forecast and warning information that are aimed at protecting lives and property. In some cases, its websites have gone down because of severe weather, unable to handle the demand.
Other agency systems, including information and data streams that deliver weather modeling data to broadcast meteorologists and commercial users, have suffered periodic outages. Many of these issues have revolved around problems with the agency’s “telecommunications gateway,” which is a portal through which computer models and other weather data the agency gathers are disseminated.
Now, during a year that featured record California wildfires and the busiest Atlantic hurricane season, the National Weather Service says it has run out of bandwidth and is seeking to throttle the amount of data its most demanding users can access. The National Weather Service, which is part of the National Oceanic and Atmospheric Administration (NOAA), announced the proposed limits in a memo dated Nov. 18.
“As demand for data continues to grow across NCEP websites, we are proposing to put new limits into place to safeguard our web services,” the memo stated, referring to the National Weather Service’s National Centers for Environmental Prediction. “The frequency of how often these websites are accessed by the public has created limitations and infrastructure constraints.”
The National Weather Service’s proposed remedy is to limit users to 60 connections per minute on a large number of its websites that provide weather observations, forecasts, warnings, computer model data, air quality information, aviation weather support and ocean conditions. The agency did not immediately reply to requests for comment.
According to companies that draw large amounts of this data, the proposed limit will hinder the services they provide to customers. The possible negative effect on forecasts has also raised concerns among congressional lawmakers.
Jonathan Porter, a vice president and general manager at the private forecasting firm AccuWeather, warns that the agency’s proposed solution would harm the timeliness and accuracy of forecasts and severe weather warnings. He said the collection, processing and distribution of weather information are the agency’s “most important services.”
“Limiting the amount of data that can be accessed conflicts with this vital mission and will negatively, and possibly catastrophically, impact individuals and businesses that rely on this data to make critical, lifesaving decisions when seconds count and lives are on the line,” Porter wrote in a statement.
Consumers who get their information primarily from weather apps on their phones also would be affected by this proposal since the forecasts and weather alerts they receive are based on computer modeling output and radar data, much of which comes from the National Weather Service.
“For private industry, it’s a huge impact, especially for users that are adjusting a lot of National Weather Service models,” said Matt Rydzik, applications developer for Commodity Weather Group, which serves clients in the agricultural and energy market.
The Commodity Weather Group operates the website StormVistaWxModels.com, a hub of computer model and forecast data that customers rely on to make what are in many cases financially costly decisions. Weather hobbyists also pay to access the site’s rapidly updated modeling data for predicting weather.
The StormVista site draws in data from more than a dozen models, including the American GFS and European model, and Rydzik said simply pulling data from one of them would potentially exceed the National Weather Service’s data limit. He called the effect of the National Weather Service proposal potentially “devastating” because it would force his company to reduce services.
“I’m assuming it would be impossible to bring in half the model runs,” Rydzik said. “It’s like a store rationing a family to one loaf of bread per week.”
Levi Cowan, who runs the weather model website TropicalTidBits.com, said the data limit would also result in delays in the delivery of model information by up to “a few hours.” In a worst case scenario, he said, this would compromise the timely delivery of vital data to consumers during extreme weather events.
Rydzik said that the Commodity Weather Group is exploring how it might work around these issues but that it may be costly.
“It is not clear why the NWS is considering these harmful bandwidth restrictions given the massive scalability of content delivery network technology, cloud infrastructure and other technology solutions that are currently available,” AccuWeather’s Porter said. “It’s truly unfortunate that the NWS apparently does not recognize that this proposal is 100% contrary to its mission and its obligation to the American people.”
Porter said the amount of data available to predict the weather continues to grow, making it more crucial than ever that the National Weather Service ensure the timely flow of information.
“It is pretty obvious that the National Weather Service has failed to keep up with IT infrastructure needs,” said Troy Kimmel, a meteorologist and lecturer at the University of Texas at Austin who closely monitors Weather Service website performance. “They don’t need to be shutting people off. They need to increase the bandwidth.”
Many private weather companies develop their own computer models for forecasting but still use National Weather Service data. Boston-based ClimaCell, which specializes in short-term forecasting, says that it would be affected by this proposal, but that the bigger picture concerns how NOAA would fall behind its competitors in Europe and elsewhere. Rei Goffer, a co-founder and chief strategy officer of the company, said the move could hurt the National Weather Service’s competitiveness if it is not limited to the short term.
“Hopefully, NOAA is using this temporary limitation to set up a state-of-the-art delivery platform,” Goffer said in a statement. “Yet, if this will be longer than a short limitation, NOAA will very quickly lose ground to competing agencies (namely UK Met and ECMWF) as the go-to source for governmental weather data.”
The Weather Company, which is owned by IBM and is one of the largest providers of weather information, declined to comment on the proposal but said it was submitting feedback directly to NOAA and Congress.
- - -
The National Weather Service held a public forum Tuesday to discuss the proposal and answer questions. When asked about the investment in computing infrastructure that would be required for these limits to not be necessary, agency officials said a one-time cost of about $1.5 million could avert rate limits. The NOAA budget for fiscal 2020 was $5.4 billion.
The officials said that senior management at the National Weather Service was aware of the relatively small cost of addressing the issue but that the agency faced “competing priorities.” Officials said they did not know whether members of Congress were aware of the issue.
Maria Cantwell, D-Wash., ranking member of the Senate Commerce Committee, which oversees NOAA, said a request to upgrade the National Weather Service’s computing infrastructure probably would find bipartisan support.
“From wildfires in Washington to hurricanes on the Gulf Coast, seconds count to save lives and property, and weather data plays a critical role in getting our emergency managers and first responders the information they need,” she said. “The United States should be striving to be the best in the world when it comes to weather data and forecasts, and with everything we’ve seen this year, a request to upgrade servers at the National Weather Service would find support on both sides of the aisle in Congress. Telling people to limit their use of this critical data is not an acceptable answer.”
The House Science, Space, and Technology Committee “is aware of the proposal” and monitoring its potential impacts, according to committee staffers. “We are looking into how these proposed restrictions could impact NOAA’s ability to ensure free and open public access to the Agency’s data and models,” a spokesperson said.