“The men and women of America’s intelligence agencies are overwhelmingly dedicated professionals,” Ron Wyden said on Wednesday, before proceeding to excoriate their bosses in front of the Senate Intelligence Committee. Addressing Director of Intelligence James Clapper, CIA Director John Brennan and FBI Director James Comey, Wyden continued,

“They deserve to have leadership that is trusted by the American people. Unfortunately, that trust has been seriously undermined by senior officials’ reckless reliance on secret interpretations of the law and battered by years of misleading and deceptive statements that senior officials made to the American people. These statements did not protect sources and methods that were useful in fighting terror. Instead, they hid bad policy choices and violation of the liberties of the American people.’

With the future of the surveillance programs disclosed by Edward Snowden still uncertain, the ball is very much in Congress’s court. Specifically, in the Intelligence And Judiciary Committees, which have critical oversight roles over the National Security Agency and others. The stakes are higher than the individual programs revealed by Snowden, however. In question is Congress’s ability to act as an effective watchdog over an expanding national security state.

Wednesday’s hearing nicely showcased the two major hurdles to congressional oversight. The first, as Wyden argued, is senior intelligence officials. The second is the congressional committees, which face an institutional mismatch with the intelligence community and whose members often seem more committed to protecting, rather than scrutinizing, the agencies they are tasked with overseeing.

First, a brief history. After revelations about abuses by the CIA and other agencies in the 1970s, Congress struck what Stephen Vladeck, a professor of law at American University, calls a “grand bargain,” to accommodate the paradoxical need to submit secret programs to democratic oversight. New legal constraints on intelligence activities would be enforced not in public but instead behind the veil by the intelligence committees and the Foreign Intelligence Surveillance Court (also known as the FISA court). As Colorado Senator Mark Udall said at the Intelligence Committee hearing, “This committee was created to address a severe breach of trust that developed when it was revealed that the CIA was conducting unlawful domestic searches.”

The Snowden leaks indicate the bargain has broken down. “In essence, the delicate balance Congress sought to strike thirty-five years ago now appears to be tipped, rather decisively, in favor of the intelligence community,” Vladeck explained in an e-mail.

Intelligence officials have done some of that tipping themselves, by withholding information from the public and lawmakers. On Wednesday, Wyden said the committee had been “stonewalled” by intelligence officials; indeed, none of his questions received direct answers, although Wyden did receive promises from officials to get back to him by specific deadlines. Wyden also cited several incidents in which officials had given inaccurate testimony in public hearings. Last March, for example, James Clapper told Wyden that the NSA did “not wittingly” collect data on American citizens, a claim we now know from the Snowden leaks to be false.

In some cases, officials may not be telling legislators anything at all. “There are certain things that the committees are simply not going to find out about unless they’re briefed,” said Representative Adam Schiff, a California Democrat on the House Intelligence Committee. According to law, the executive branch must keep the intelligence committees “fully and currently informed” of any intelligence activity, including significant anticipated activity. But recent history suggests that even committee chairs have not been kept abreast. Senator Dianne Feinstein, who chairs the Senate Intelligence Committee and has become one of the NSA’s closest congressional allies, has admitted that her committee “was not satisfactorily informed” about certain surveillance activities, in particular the tapping of German chancellor Angela Merkel’s cell phone.

The reauthorization of Section 215 of the Patriot Act, which the NSA says provides legal grounds for collecting Americans’ phone records in bulk, presents another case in which Congress appears to have had insufficient information to determine whether intelligence activities were lawful. In a report released last Friday, the Privacy and Civil Liberties Oversight Board explained that the FISC did not articulate the legal basis for bulk collection until last August—years after Congress extended Section 215 in 2010 and 2011. This indicates that lawmakers may not have been properly informed of how the statute was being applied when they extended it. Ultimately, the board argued that Section 215 is not written in a way that justifies bulk data collection at all, but it said that even if the language were ambiguous, some members of Congress “may have been prohibited from reading” critical documents before voting to maintain Section 215.

One explanation for these gaps is the institutional mismatch between the intelligence community and the congressional committees. “The intelligence committees are small, the staff is small, the agencies themselves are behemoth,” said Schiff. In the House, members are not permitted to have their own staff on the committee, and some have described feeling inadequately prepared to question intelligence officials. “You don’t have any idea what kind of things are going on. So you have to start just spitting off random questions: Does the government have a moon base? Does the government have a talking bear? Does the government have a cyborg army?” Representative Justin Amash said in October at a conference hosted by the Cato Institute.

The administration says Congress is duly informed, while other lawmakers have suggested it’s their colleagues’ own fault if they aren’t up to speed. Clapper reaffirmed promises of greater transparency on Wednesday, but critics remain skeptical. “I don’t think this culture of misinformation is going to be easily fixed,” Wyden warned. Representative Schiff told me that he expects the mismatch in resources to continue to impact the committees’ oversight ability. “I’d like to see our capacities augmented in the intelligence committee, but at a time of dwindling legislative budgets, I’m not sure whether that will take place,” he said.

Congress could reassert some of its own authority by including more members in the group briefed on significant intelligence activity, for example; by shortening the authorization period for laws like the Patriot Act to spur more frequent debate; by imposing a rule of lenity on the FISA court, so that the administration would have to receive congressional approval in ambiguous cases, thus preventing the FISA court from creating its own novel interpretations of law; and by making sure FISA judges hear adversarial opinions from civil liberties and technology experts. Whether Congress has the will to strengthen its own hand is the big question. The fact that the chairman of the House Intelligence Committee, Mike Rogers, believes that “you can’t have your privacy violated if you don’t know your privacy is violated” does not exactly inspire confidence.

These questions about oversight can and should be separated from the debate about the legality and effectiveness of individual surveillance programs. “It may well be, at the end of the day, that the programs the committees are overseeing are legal, but meaningful oversight presupposes that the overseers will be the last, not the first, to reach that conclusion,” said Vladeck. The decision about whether the government or anyone else may continue to sweep up and search our phone records is important. Without addressing broader oversight issues, however, we may barely scratch the surface of a much-needed conversation about balance of power in the post-9/11 era.