The push to remove JPMorgan Chase CEO Jamie Dimon and other financial-sector executives from the Federal Reserve Boards of Governors came inside the walls of the Fed on Monday, as noted economist Simon Johnson presented officials there with a petition and urged them to change the structure of the important boards.

At the twelve regional Federal Reserve banks, there are nine-member boards of directors. Six of the seats are selected by banks from the region—three directors to represent their interests, and then three directors, picked by the banks, that will allegedly represent “the public’s interest.”

Dimon sits on the Federal Reserve Bank of New York and has become a poster boy for activists seeking to keep bankers off the Boards of Governors. The petition that Johnson, a professor at the Massachusetts Institute of Technology, presented to Federal Reserve staffers has nearly 38,000 signatures, and asks that he resign or be removed.

The essential conflict of interest is that the Federal Reserve is charged with maintaining the safety and soundness of Wall Street banks, and executives at those institutions often resist such changes in the name of riskier gambles and bigger profits. Moreover, in recent years the Fed has handed out over $4 trillion in zero-interest loans to many of those banks. (JPMorgan Chase received $390 billion in emergency funds during the bailouts, and $29 billion to buy Bear Stearns).

“Frankly, I think Jamie Dimon should have resigned in the spring of 2008 when JPMorgan Chase acquired Bear Stearns with financing provided in part by the Federal Reserve,” Johnson said on a conference call shortly after his meeting. “I think to any outsider, anyone with knowledge of how corporate governance operates in general in the United States, or best practices around the world, this looks like a related-party transaction, and typically you would step down from the board of directors when something like this was in the works.

“It is surprising and very uncomfortable to many people, including many people who are close to the Federal Reserve system, that Jamie Dimon has continued in this position.”

But while Johnson was happy that Federal Reserve staffers, including General Counsel Scott Alvarez, met with him, he wasn’t necessarily encouraged by the discussion.

“I have not felt optimistic about either Jamie Dimon resigning or the Federal Reserve, both at the New York level or the Board of Governors level—changing its policies in a substantial way, in a way that would help to restore confidence in the integrity of the system,” Johnson said. “I’ve not felt that optimistic for quite some time, and there’s nothing that happened today, unfortunately, that changed my lack of optimism.”