DevHeads.net

bad udp cksum

Hi,

Recently I'm noticing an interesting issue.
My CentOS servers are trying to send logs to a logging server via 514/udp,
however I'm not receiving anything.

I did the following on CentOS
*tcpdump -vvv -nn udp -i esn160 port 514*

In another session on the same server:
*nc syslog-server -u 514*

tcpdump started to show me messages like:
*[bad udp cksum 0x3ce9 -> 0xb0f5!] SYSLOG, length: 172*

After some research I disabled TCO (ethtool -K ens160 tx off rx off), now
tcpdump shows:
*[udp sum ok] [|syslog]*
And I'm also receiving the logs. Yay!

I have the same issue on multiple servers with CentOS versions 7.3.1611,
7.4.1708 & 7.5.1804. I'm having the issue no matter if it is a physical or
virtual server (they also use different hardware and NIC's)

Of course I could run ethtool every time a server starts but I think there
should be a better solution. Also I enjoy having TCO giving some rest to
the CPUs.

Did anyone find a solution for this?

Thanks!
Laszlo

Comments

Re: bad udp cksum

By Eliezer Croitoru at 09/06/2018 - 06:51

I have seen such and issue but it depends on the environment structure.
I have seen it mostly on VM's and it was resolved.
The hardware and software details of the setup might help to understand
what's causing it.

Eliezer

Danielisz

Hi,

Recently I'm noticing an interesting issue.
My CentOS servers are trying to send logs to a logging server via 514/udp,
however I'm not receiving anything.

I did the following on CentOS
*tcpdump -vvv -nn udp -i esn160 port 514*

In another session on the same server:
*nc syslog-server -u 514*

tcpdump started to show me messages like:
*[bad udp cksum 0x3ce9 -> 0xb0f5!] SYSLOG, length: 172*

After some research I disabled TCO (ethtool -K ens160 tx off rx off), now
tcpdump shows:
*[udp sum ok] [|syslog]*
And I'm also receiving the logs. Yay!

I have the same issue on multiple servers with CentOS versions 7.3.1611,
7.4.1708 & 7.5.1804. I'm having the issue no matter if it is a physical or
virtual server (they also use different hardware and NIC's)

Of course I could run ethtool every time a server starts but I think there
should be a better solution. Also I enjoy having TCO giving some rest to
the CPUs.

Did anyone find a solution for this?

Thanks!
Laszlo

Re: bad udp cksum

By =?utf-8?B?SsO8c... at 09/06/2018 - 07:59

Well, it is in general a good idea to disable all the fancy nic feature like tso, gro, rx/tx offload if strange things happen

-----Ursprüngliche Nachricht-----
Von: CentOS <centos- ... at centos dot org> Im Auftrag von Eliezer Croitoru
Gesendet: Donnerstag, 6. September 2018 13:51
An: 'CentOS mailing list' < ... at centos dot org>
Betreff: Re: [CentOS] bad udp cksum

I have seen such and issue but it depends on the environment structure.
I have seen it mostly on VM's and it was resolved.
The hardware and software details of the setup might help to understand what's causing it.

Eliezer

Hi,

Recently I'm noticing an interesting issue.
My CentOS servers are trying to send logs to a logging server via 514/udp, however I'm not receiving anything.

I did the following on CentOS
*tcpdump -vvv -nn udp -i esn160 port 514*

In another session on the same server:
*nc syslog-server -u 514*

tcpdump started to show me messages like:
*[bad udp cksum 0x3ce9 -> 0xb0f5!] SYSLOG, length: 172*

After some research I disabled TCO (ethtool -K ens160 tx off rx off), now tcpdump shows:
*[udp sum ok] [|syslog]*
And I'm also receiving the logs. Yay!

I have the same issue on multiple servers with CentOS versions 7.3.1611,
7.4.1708 & 7.5.1804. I'm having the issue no matter if it is a physical or virtual server (they also use different hardware and NIC's)

Of course I could run ethtool every time a server starts but I think there should be a better solution. Also I enjoy having TCO giving some rest to the CPUs.

Did anyone find a solution for this?

Thanks!
Laszlo
financial.com AG

Munich Head Office/Hauptsitz München: Georg-Muche-Straße 3 | 80807 Munich | Germany | Tel. +49 89 318528-0 | Google Maps: <a href="https://goo.gl/maps/UHwj9" title="https://goo.gl/maps/UHwj9">https://goo.gl/maps/UHwj9</a>
Frankfurt Branch Office/Niederlassung Frankfurt: Messeturm | Friedrich-Ebert-Anlage 49 | 60327 Frankfurt am Main | Germany | Google Maps: <a href="https://goo.gl/maps/oSGjR" title="https://goo.gl/maps/oSGjR">https://goo.gl/maps/oSGjR</a>
Management Board/Vorstand: Dr. Steffen Böhnert | Dr. Alexis Eisenhofer | Dr. Yann Samson
Supervisory Board/Aufsichtsrat: Werner Engelhardt (Chairman/Vorsitzender), Eric Wasescha (Deputy Chairman/Stellv. Vorsitzender), Franz Baur
Register Court/Handelsregister: Munich – HRB 128972 | Sales Tax ID Number/St.Nr.: DE205370553

Re: bad udp cksum

By Simon Matter at 08/10/2018 - 02:34

You can configure the ETHTOOL_OPTS option in the ifcfg files to
automatically assign settings on startup.

Regards,
Simon

Re: bad udp cksum

By Laszlo Danielisz at 08/10/2018 - 10:48

I could, however that is going to increase the CPU load, mostly under heavy
network load.

On Fri, Aug 10, 2018 at 3:34 AM, Simon Matter <simon. ... at invoca dot ch>
wrote: