In 1961, at the pool of Miami’s Sir John Hotel, Cassius Clay held an underwater power-punching demonstration for Flip Schulke, a photographer on assignment from Sports Illustrated. Clay had won a gold medal the previous summer, in the light-heavyweight division, at the 1960 Olympics. After viewing a sample of the photographer’s earlier work—a collection of underwater pictures of water skiers—Clay arranged for the two to meet poolside for a shoot.

You might recognize one of the photos, even though it never was selected for publication, when a profile of Clay ran in the September issue of Life. (Sports Illustrated turned Schulke’s pool idea down first.) In it, the 19-year-old boxer—who would later rename himself Muhammad Ali—squats underwater in an orthodox fighter’s stance, his mitts up and clenched. “He was holding his breath all this time and not making any movement,” Schulke remembered. “There he was, standing on the bottom of the pool. I mean, that’s very hard to do, and he’s in a perfect boxing pose.” Clay, as delicately rendered by Schulke, looks like a bodybuilder flexing on stage: His face remains stoic, relaxed while his body stays tense.

When I think of the illustrious boxer, who died last month, my memory searches for a still image. One of the first that surfaces is Schulke’s portrait, what would later be known as Ali Underwater. Somewhere not far behind is Neil Leifer’s celebrated photo, from 1965, of Ali’s first-round “phantom punch” knockout of Sonny Liston. There, the self-proclaimed “greatest” is barking at, and towering over, a fallen Liston, who lies supine and dazed in the middle of the ring. These two photos convey what we have come to know—to the extent that one can truly “know”—to be a shortlist of Ali’s multitudes: strength, ebullience, confidence, anger, and passion. These still images tell me something about Ali; though far from a complete story, they preserve a visual record of who he was.

* * *

Photographs, curator and author Gail Buckland argues in her recent book, Who Shot Sports: A Photographic History, 1843 to the Present, have played a powerful role in molding sports history. (The book is a companion to an exhibit at the Brooklyn Museum, which opens this month.) “When people remember moments in sports, it is often a still photograph that comes to mind,” she writes in the book’s opening pages. Much like the larger, modern project of photography has demonstrated its capacity to crystallize a moment in time, sports photography, more acutely, preserves human drama long after game’s end. “When the action stops, the still photograph remains,” Buckland writes. Here, she offers a tacit acknowledgment of photography’s potential to alter what we remember.

Consider Andrew D. Bernstein’s intimate off-the-court photo, which could be called Kobe on Ice—collected in Who Shot Sports—in which the Lakers star is shown partaking in a pregame locker-room ritual. Bryant sits in a trance akin to Rodin’s Le Penseur. His two feet are submerged in an oversized cooler of ice water, his right index finger dunked in a small mug. The photo could serve as an avatar of Bryant’s tenure in the NBA: Here he appears solitary, introverted, and contemplative—a reflection of his earlier, younger self, no doubt—but also tired and physically aging. The moment, according to Bryant’s former coach Phil Jackson, is from 2010, one year after Bryant scored 61 points in a game at Madison Square Garden. Yet, the photograph also imposes itself onto Bryant. Somehow, it represents more than the moment it enshrines; it channels a broader understanding of his legacy.

This feeling—when a picture reaches beyond the moment it captures—surfaces while looking at numerous images in Who Shot Sports. Take, for instance, a black and white photograph of boxer Jack Dempsey and Yankees pitcher Babe Ruth, clad in silk robes and slippers, eating breakfast in Ruth’s Upper West Side apartment inside the hotel Ansonia. Both men grimace, staring at the camera as if surprised; the lefty, Ruth, raises his spoon from his bowl with his pitching hand while Dempsey, his bare legs crossed under the white table cloth, pushes food around on his plate with his right. There is an awkwardness to the encounter; the photographer levels the camera’s vision as though it’s seated, interrupting an otherwise quotidian morning of decadent dining. By the looks on the subjects’ faces, it probably is. The aesthetic quality of the photo—its nuance, surprise, and intimacy—produces a more robust, complicated portraiture of two heralded athletes. This is who they are when we aren’t looking.

That athletes are captured from various angles is one of sports photography’s most longstanding virtues. Photographers play a game among themselves called “find the best place to put your camera,” Buckland writes. One such practitioner, Lucy Nicholson, a British journalist and photographer known for writing her own captions—in clinical, subdued prose—likened the competitiveness of shooting a sporting event—pushing, jostling for position—to the action on a football field. “At the end of some classics [Super Bowl, Olympics], there are some real scrums,” she tells Buckland, but “there is a camaraderie in sports that I really like.”

We see the unconventional vantage points on display in Who Shot Sports. In one shot from 2007 titled Equestrian, Bob Martin, an English photographer who has spent more than 30 years specializing in shooting action, shows us the power and speed of eight thoroughbreds landing after clearing a hedge. From the ground, the blades of grass and dirt being kicked up by the steeds give the photo its force. In another crafty photo, Sports Illustrated’s Peter Read Miller shoots the gymnast Gabby Douglas from above, during her balance-beam routine at the 2012 Olympic trials in San Jose, California. The symmetry of Douglas’s body is arresting: She splays out evenly like a weathervane, looking upward, her head centered over the four-inch beam. From Miller’s perch, we see her bandaged left ankle and chalk on the bottom of her opposite heel as she kicks her right leg back. Powdery footprints on the tan beam and blue mat below allude to what came next. (She stuck the landing.) Peering down, the ground takes on a deceptive, paradoxical property: It’s as if we are staring upward at Douglas pirouetting under a cloudy sky.

The photographer’s vantage point might be considered the main focus of Who Shot Sports—understanding the perspective of the person behind the camera. Buckland homes in on the relationship between photographer and subject quite often throughout the book. She fills the pages with colorful anecdotes and names, as if on a magazine’s contributor page. Additionally, she provides a detailed timeline of both the people and technological advances that have fueled sports photography’s evolution. She reasons: “If the photograph is well known, the photographer should be, too.”

There is something fascinating in the premise: in refocusing our attention to the person behind the camera—the person we don’t see—Buckland bridges the individual’s story to the images that have shaped our memory. We imagine the ways in which a story, as represented visually, can impact—haunt, even—the storyteller.

* * *

While reading Who Shot Sports, I kept returning to another of Schulke’s gorgeous black and white stills—which also didn’t make the pages of Life. In it, Ali rises from the water: His damp, black curls slope across his forehead, the wetness trickling down over his ears, traversing his hulking, uninhibited shoulders. The water glistens as it rushes across his chest, jetting out from the rocky thews of his abdomen like a waterfall.

Buckland mentions in an aside that the photo came about after Schulke’s plans to take the young boxer—who was still relatively poor at the time—shopping were thwarted by racist shopkeepers. “Even a big SI budget didn’t open the doors of prejudice,” she writes. This detail struck me as significant, though it was not immediately discernible—at least not explicitly in the text—as to why. We don’t find out that Schulke is perhaps one of the most notable photographers of the civil-rights era.

Three years before photographing Ali in the pool, Schulke befriended the Rev. Dr. Martin Luther King Jr., while on assignment from Ebony. (“Outside of my immediate family, [King] was the greatest friendship I have ever known or experienced,” he would later say.) During his long tenure as a freelance photographer, he took more than a half-million photographs—11,000 of which were of King and other activists fighting systemic racism—which have appeared in numerous publications and collected in several books.

Perhaps a detailed biography of each subject in an undertaking of this scale—there are 250 photos in Who Shot Sports, covering more than 150 years—is too much to ask for. Despite Buckland’s relatively small representative sample of photos—and even smaller considering the amount of shots it took to capture these moments with such clarity—there is little depth to the photographers’ stories. The pictures themselves provide an account of their makers’ visions, imaginations, and points of view. But the images also take up space, and they overshadow the names. In this way, the photographer remains—true to photography’s tradition—present, but unseen.

Still, Buckland’s project is noteworthy in its presentation of a sophisticated idea—that one subject’s story (what’s in the frame) can influence the next (what’s left out). That history and art coincide, breathe life into each other. That there is a history left undocumented, waiting for its story to be told, as in literature.

After all, it was Gustav Janouch who told Franz Kafka, “The necessary condition for an image is sight.” And it was Kafka who answered with a smile: “We photograph things in order to drive them out of our minds. My stories are a way of shutting my eyes.”