First, when we talk about latency here, we mean the complete „key to sound latency“, i.e. the time between pressing a key and the start of the sound. It’s important to distinguish this from the pure audio output latency, which is the time between handing over audio data to Android until the sound can be heard. If you see a one digit latency number, that’s typically either pure output latency or the size of some audio buffers. Good „key to sound latency“ for virtual instruments starts below 50ms. According to http://www.speech.kth.se/music/5_lectures/askenflt/measure.html, the „key to sound latency“ of a real piano from touching a key until the string vibrates is about 20 to 30 ms (staccato forte case) due to the mechanics involved.
Up until Android JellyBean/4.1, Android itself had a serious latency problem (see http://code.google.com/p/android/issues/detail?id=3434). Actually, it was a problem with pure audio output latency and with process priorities/scheduling. Since JellyBean, Android can support low latency – if the app developer and the hardware manufacturer have done their homework. For my music apps, I did. They all support the low latency features of JellyBean and later. However, most android device manufacturers did not. This article might be of interest in this context: Why Mobile Low-Latency is Hard, Explained by Google as well as this video: Google I/O 2013 – High Performance Audio.
So how can you reduce the audio latency? Besides using Midi over USB (that’s actually faster than using the touch-screen), the best answer is to use an Android device that supports low latency.
The best low latency Android device I know of is the Galaxy Nexus. It’s outdated, but none of the more recent devices matches its latency of about 60ms.
The Nexus 7 (2012) is on a second tier, reaching about 90-110ms. As far as I know, the Nexus 4, the Nexus 10 and the Sony Xperia Z/ZL should be in this tier too, but that’s hearsay. Please, be aware that the Nexus 4 has a problem with USB-Host/OTG.
First tests show that the latency of the Nexus 7 (2013) is roughly comparable to the Nexus 7 (2012) [Read More].
As far as I know, most other devices have worse latency, often in the range of 150-300ms.