Other towers can still passively track your signal. You also have to communicate with more than one tower during handoff between cells. This is a capability mandated by E911.
To locate a mobile telephone geographically, there are two general approaches. One is to use some form of radiolocation from the cellular network; the other is to use a Global Positioning System receiver built into the phone itself. Both approaches are described by the Radio resource location services protocol (LCS protocol).
Look at the tower. It would have at least 3 antennas giving at least a 120 degree segment. As soon as a phone connects to another one you would have a line (or more like an ellipsoid) between the two towers. Even if the phone just lies on the desk it could switch towers, eg if the current one has too many clients, or the signal quality is pretty equal between them.
NB I worked in a such place, my phone would be in low 40-50% at the end of the work day, despite being able to endure two days easily if I would be at some other place.