Cisco Firewall :: 5520 - Inside Server To See Actual Outside Host Source IP In Udp Packet

I have a 5520 in production at a customer's site between an outside 802.11 network and an inside server.   The server can get to outside hosts OK, and the traffic is being NATed  properly, and sockets initiated by the server on the inside can pass data both ways, but I need to allow outside hosts the ability to send  'announcement' UDP packets to the inside server.  I thought this might be an  outside-NAT-required issue to get the traffic routed, but I need the inside server to see the  actual outside host source IP in the UDP packet, so I basically set the  outside host up similar to the inside host, just without the NAT table on the firewall -- it's subnet is outside the  destination (inside server) subnet, and its gateway is the outside  interface of the ASA, the same way the inside server is able to get to  hosts outside.  The firewall should just route the packet with a destination of the inside subnet once it sees that it hits a 'permit' ACL.
 
I have the appropriate ACL's set up, and when I do 'show access-list' I  see policy hits for the 'permit' statements where the outside host is  generating the announcement and it's hitting the ACL.  I even duplicated  the ACL into list 101 and 102, and applied 101 for inbound traffic on  the outside int, and applied 102 for outbound traffic on the inside int,  and I'm seeing policy hits on both permit statements outside and  inside, so it looks like the traffic is being passed on to the inside  interface and permitted, but the server isn't seeing the packets.
 
I can ping the outside interface from the outside, but cannot ping the  inside interface or any inside hosts from the outside, even though I  have 'permit icmp any any' enabled on the ACL on both ints. When I  remove the firewall and put the outside clients on the same subnet, the server sees the packets just fine.
 
I set up the same scenario in my lab with an ASA 5505, with the same results.  Below is the running config from the 5505 in the lab.  The production firewall is running a slightly older version of ASA, so I made the configuration as basic as possible on the 5505 to match the config in the field:
 
: Saved
:
ASA Version 8.3(1)
!
hostname ciscoasa
enable password Guh9Xxhb9mcC8lV1 encrypted
passwd 2KFQnbNIdI.2KYOU encrypted
names
!
interface Vlan2
description Outside WAN Interface
nameif outside
security-level 0
ip address 192.168.10.1 255.255.255.0
!
interface Vlan3
description Inside LAN Interface
nameif inside(code)