“He [second baseman Frank Ward] is a lively and amusing coacher and a daring baserunner, and he puts lots of life into his work on the keystone bag.” Galveston Daily News, February 2, 1890, p4
Previous earliest use (Dickson Baseball Dictionary, 3rd edition, 2009):
1ST USE. 1906. “[Jake] Stahl on first and [Lave] Cross on third…will brace up the youngsters around the keystone bag” (Chicago Tribune, Apr. 8; Peter Morris).