If there's no form of NAT or transport later processing along your path between endpoints you shouldn't have an issue. But NAT and transport and application layer load balancing are very common on the net these days so YMMV.

You might have more luck with an IPv6 packet.