The handshake for establishing a bittorrent connection with a peer literally starts with the string "Bittorrent protocol". I don't think it's that hard for ISPs to detect that. Bittorrent encryption prevents that but it's not used everywhere.