It is not acceptable that thousands of Texans spent hours trying to reserve appliance rebates on Wednesday, April 7, only to be met with dropped calls and an unavailable online reservation system. It is not acceptable to me or the many Comptroller employees who worked diligently to ensure mechanisms were in place to accommodate the extremely high demand we expected. And it certainly isn’t acceptable to the citizens of our state. We all deserve real answers.
With high-value rebates, a large population and limited funds, we knew not everyone interested in the program would be able to reserve rebates. We closely watched similar rebate programs in other states. Within days of seeing the experiences of Iowa and Minnesota, whose systems were overloaded, we were on the phone requiring increased safeguards from Helgeson Enterprises, the company contracted through a competitive bid process to manage our rebate fulfillment. We accurately projected the estimated traffic we thought we would receive in the first few hours. We looked closely at the contractor’s system bandwidth and infrastructure. We provided clear steps they needed to take at a minimum to prepare for Texas.
On our end, we immediately recognized the need to create our own Web front door for the contractor’s online system to provide program information and updates. We took the steps needed to handle millions of hits concurrently to our Texas Powerful Smart site at www.TexasPowerfulSmart.org. We did it right and our Web front door handled about 40 million hits in less than 12 hours without fail. This allowed us to provide real-time updates about the contractor’s system and direct people to our e-mail subscription service and Twitter feed for updates to keep them informed of rebates remaining and other important details.
Prior to April 7, at our request the contractor did a test run of their system and stated that the online reservation system should be able to handle 25,000 users at a time. The wait time for those trying to access the system online was estimated by the contractor at 3 minutes. Dropped calls were never acceptable to us and no one should have been kicked out in the middle of an online reservation.
However, problems with their online reservation system began immediately. With their online system unavailable, the phones were completely overloaded. Our agency took thousands of calls as well. Through a conference call bridge set up from early morning Wednesday, we demanded updates from our contractor on the system status to let people know via our site and other communication channels.
By mid-morning, the contractor reported their system may have been affected by a Web attack, which consumed about 75 percent of their capacity.
The online reservation system was brought back online at 12:15 p.m. and reservations started flooding in. Just after 2 p.m. all rebate reservations were taken and the waitlist was opened. It filled by 6 p.m. However, many still reported problems with the system.
To the thousands of frustrated Texans who experienced problems: we are as mad as you are. We are gathering information to identify the full scope of the problem. We have received feedback from more than 15,000 of you and we are taking action.
Before the end of the day on Wednesday, we sent a letter to Thomas Helgeson, president of Helgeson Enterprises, expressing our dissatisfaction with the company’s performance and requesting specific sets of data that will help us determine whether the company is meeting its contractual obligations for items such as Web site uptime and answer time for phone calls. We also requested a specific description of the steps Helgeson is taking to avoid additional problems with rebate fulfillment.
Not one dollar has been paid to Helgeson. Payment is based on the number of rebate claims processed. This means no payment to Helgeson will be made until Helgeson begins to issue rebate checks to Texans, and we have a team actively monitoring this entire process.