“…. he [the umpire] decided the player not out, though touched with the ball while off his base, because he thought that Crane’s action when he first touched him caused Littlewood to lose his balance … There is altogether too much of this pushing off the base style of play in vogue.” The Brooklyn Union, August 19, 1865, p1
Previous earliest use (Dickson Baseball Dictionary, 3rd edition, 2009):
1ST USE. 1872. “The Atlantics…would have made another [run] had not Thake got caught off his base” (Brooklyn Daily Eagle, July 30; OED).
