Monitoring 10 min read
SNMP and Syslog: Giving the AV Network a Memory
Use polling, traps and timestamped logs to diagnose AV failures before the evidence disappears.
Why an AV engineer should care
A decoder goes black twice a week. Rebooting fixes it. Everyone blames the network, but the switch looks perfectly healthy when you inspect it.
The problem is not necessarily the network. The problem is that you arrived after the evidence disappeared.
SNMP and syslog let switches—and some AV devices—record what happened before anyone opened the support ticket. Used properly, they can reveal a marginal fiber link, exhausted PoE budget, congested port, overheating switch or endpoint that keeps restarting.
Today’s objectives
By the end of this lesson, you should be able to:
- Explain agents, managers, MIBs and OIDs without mystical hand-waving.
- Distinguish SNMP polling, traps and syslog.
- Read traffic, error and discard counters correctly.
- Define a practical monitoring package for an AV network.
- Secure SNMP without becoming a full-time network administrator.
1. What SNMP actually does
SNMP means Simple Network Management Protocol. The name is misleading: the protocol is fairly simple, but the surrounding terminology is impressively determined to prevent human understanding.
There are two main participants:
- The agent runs inside the switch, UPS, PDU or AV product.
- The manager, commonly called an NMS, asks agents for information and saves the answers.
An NMS might be PRTG, LibreNMS, SolarWinds, Zabbix or another monitoring platform.
The agent exposes information through a MIB, or Management Information Base. Think of a MIB as a structured dictionary describing what information a product can report.
An OID, or Object Identifier, is the address of one particular item in that dictionary.
For example, an OID might represent:
- Port operating state
- Bytes received
- Input errors
- Temperature
- System uptime
- PoE power consumption
The standardized Interfaces MIB includes familiar measurements such as 64-bit octet counters, errors, discards and multicast packets. Vendor MIBs extend this with manufacturer-specific information such as stack health, PoE details or optical transceiver measurements. (rfc-editor.org) Important distinction: the MIB is not the historical database. It defines what the device can report. The monitoring server builds history by asking repeatedly and storing the answers.
2. Polling: “How are you doing right now?”
During polling, the NMS periodically requests values from the agent.
A reasonable interval might be:
- 60 seconds for important AV endpoints and uplinks
- 5 minutes for temperature and other slowly changing conditions
- Faster only when there is a specific reason
The NMS can graph those answers over hours, days or months.
This is far more useful than ping. Ping tells you:
This address answered me just now.
SNMP can tell you:
The port remained up, but received 4,000 errors during the blackout, its optical receive level has been drifting downward, and the switch has not rebooted.
Counters versus gauges
A gauge is a current reading: temperature, fan speed or instantaneous PoE draw.
A counter generally increases until the device restarts or the counter wraps. Traffic counters therefore do not directly say “this port is using 600 Mbps.” The NMS calculates a rate:
Octets/s = (new octet counter − old octet counter) / elapsed seconds
Bits/s = octets/s × 8
Use 64-bit high-capacity counters on gigabit and faster interfaces. Their standard names include ifHCInOctets and ifHCOutOctets; the “HC” means high capacity. The older 32-bit counters can wrap surprisingly quickly on a busy link. (rfc-editor.org) Discard a rate sample that spans a counter reset or discontinuity rather than treating it as real traffic. Standard sysUpTime measures the SNMP management subsystem: a reset can mean the agent restarted, not necessarily the whole switch. Use vendor device-uptime measurements and correlated syslog where available to confirm a chassis reboot.
Always monitor device uptime as well. A counter returning to zero may indicate miraculous network healing—or merely that the switch rebooted.
3. Errors and discards are not the same thing
These two categories are especially useful to AV troubleshooting.
Errors mean the switch could not properly receive or transmit something. Input CRC/FCS errors commonly point toward:
- Cabling or termination trouble
- Dirty or marginal fiber
- Defective SFPs
- Electrical interference
- Failing hardware
Discards mean a packet may have arrived intact but was deliberately dropped. Common causes include:
- Congested queues
- Insufficient egress bandwidth
- Buffer exhaustion
- Policy or filtering behavior
- Bursts exceeding what the port can forward
Therefore:
- Rising errors suggest investigating the physical path.
- Rising discards suggest investigating traffic load, oversubscription and QoS.
- Neither counter, by itself, proves a root cause.
The standard interface MIB deliberately maintains separate error and discard counters. (rfc-editor.org)
4. Traps and informs: “Something just happened!”
Polling makes the manager ask questions. A trap lets the device initiate a message when an event occurs.
Useful notifications include:
- Port went down or came up
- Switch restarted
- Power supply or fan failed
- Temperature crossed a threshold
- PoE overload or denial occurred
- Stack membership changed
- Optical power crossed an alarm threshold
- Authentication failed
A trap is normally unconfirmed: the device sends it without requiring an acknowledgment. It can be lost.
An inform is a confirmed notification. The receiver acknowledges it, and the sender may retry if no acknowledgment arrives. Not every AV-adjacent product supports informs, so check the platform. The SNMP architecture formally distinguishes unconfirmed traps from confirmed informs. (rfc-editor.org) This is why a solid system uses both:
- Polling supplies state and trends.
- Traps or informs provide immediate event notification.
A trap alone can disappear. Polling alone can miss a ten-second failure between samples.
5. Syslog supplies the story
Syslog is a separate event-message system. A device pushes timestamped messages to a syslog collector.
SNMP might report:
Interface 48 is down.
Syslog might report:
Interface ethernet 1/1/48 changed to down at 14:32:07.
Depending on the switch, another message might identify a PoE fault, spanning-tree transition, authentication failure or transceiver alarm.
Syslog messages contain a severity numbered from 0 through 7:
| Number | Severity |
|---|---|
| 0 | Emergency |
| 1 | Alert |
| 2 | Critical |
| 3 | Error |
| 4 | Warning |
| 5 | Notice |
| 6 | Informational |
| 7 | Debug |
Lower numbers represent greater severity. Vendor choices about which events receive which severity can vary, so do not assume every “warning” deserves an emergency service call. (rfc-editor.org) Configure NTP before trusting any logs. If the switch, decoder and monitoring server disagree about the time, correlating events becomes detective work performed by three witnesses who own different clocks.
Also send logs to a remote collector. Local switch logs are often small and may disappear during the very reboot you are investigating.
6. What BMT should actually monitor
A practical AV monitoring standard does not need 4,000 reassuring green dots.
For every managed AV switch, start with:
- System uptime
- CPU, memory and temperature
- Power-supply and fan state
- Total PoE budget and consumption
- Uplink operating state and negotiated speed
- Inbound and outbound traffic rate
- Interface errors and discards
- Link-state changes
- Fiber DOM readings where available
For important endpoint ports, add:
- Endpoint port state
- PoE draw and PoE fault events
- Traffic rate
- Errors and discards
- Frequent up/down transitions
For multicast networks, vendor MIBs may also expose IGMP groups, querier status and multicast counters. Availability varies by platform. SNMP cannot retrieve information that the manufacturer’s agent never exposes.
Security baseline
SNMPv1 and SNMPv2c use community strings and do not provide modern authentication and encryption. Prefer SNMPv3 using authentication and privacy, commonly described as authPriv. SNMPv3’s User-based Security Model supports message authentication and protection from disclosure. (rfc-editor.org) BMT’s normal specification should require:
- A read-only SNMPv3 user
- Authentication and encryption using current algorithms supported by the product
- Access restricted to the monitoring server’s IP
- SNMP limited to the management network
- No default
publicorprivatecommunities - Write access disabled unless specifically required and documented
Monitoring generally uses UDP 161. Notifications commonly go to UDP 162. Those ports should not be casually exposed outside the management boundary.
7. Two real AV examples
Example 1: Intermittent AVoIP receiver blackouts
Several decoders occasionally lose video, but the problem is gone when the technician arrives.
Monitor the decoder ports and switch uplinks for:
- Link changes
- Traffic rates
- Errors and discards
- Switch uptime
- Fiber DOM receive and transmit power
- Transceiver-temperature and threshold alarms
- Traps and syslog events
If optical receive power approaches its alarm threshold while CRC errors rise, investigate the fiber, connectors and SFP.
If the port remains up with clean optics and zero errors, but discards spike during the failure, investigate congestion, multicast flooding and queue behavior.
If all switch evidence remains clean, that does not prove the network innocent—but it justifies moving the investigation toward the encoder, decoder, firmware or application layer.
Example 2: A PoE camera keeps disappearing
A PTZ camera randomly reboots during meetings.
Compare:
- Camera uptime, if exposed
- Switch uptime
- PoE wattage
- Port link transitions
- PoE denial, overload or restart events
- Syslog messages at the failure time
If the switch remains up but the camera uptime resets immediately after a PoE fault, the power path becomes the prime suspect.
If the entire switch uptime resets, investigate the switch power supply, UPS or upstream electrical system.
Polling might miss a very short event. The trap or syslog message may preserve it.
Practical exercise
Choose one switch uplink and one PoE AV endpoint. Record:
- Port name and its corresponding drawing/equipment label
- Link state and speed
- Current traffic rate
- Error and discard counters
- Switch and endpoint uptime
- DOM readings for fiber, if applicable
- PoE draw for the endpoint
During an approved test window, disconnect and reconnect the endpoint. Confirm whether the monitoring system receives a link notification and a syslog message.
The real test is not whether the dashboard turns red. It is whether someone can reconstruct the event afterward.
Compact glossary
- SNMP: Protocol for reading and sometimes changing managed-device information.
- NMS: Network Management System—the collector and dashboard.
- Agent: SNMP software running on the monitored device.
- MIB: Definition of the managed information a device can expose.
- OID: Address of one specific managed value.
- Poll: Manager-initiated request for current information.
- Trap: Unacknowledged device-initiated notification.
- Inform: Acknowledged notification.
- Syslog: Timestamped event-message protocol.
- DOM: Digital Optical Monitoring—optical power, temperature, bias and related SFP measurements.
Review questions
- What is the difference between a MIB and an OID?
- Why should traps not replace polling?
- Which three evidence sources are most useful during an intermittent AV outage?
Answers
- A MIB defines the names and structure of available information; an OID identifies one particular value within that structure.
- Traps can be lost and provide events rather than continuous state or trends. Polling verifies current conditions and creates history.
- Time-aligned SNMP polling history, traps/syslog messages and the observed AV symptoms.
Where we are headed
This lesson adds operational visibility to the networking foundation we have built. Lesson 15 will cover certificates and PKI from the AV side: why nearly every product says “this website is not trusted,” what the browser is actually objecting to, and how to correct it without teaching users to ignore security warnings.
Set up your own SNMP lab
Try read-only monitoring on a spare switch with LibreNMS. A companion to this lesson, not another lesson in the course.
Open the SNMP lab guide →Practical example: A tiny SNMP test before building a dashboard
Start with a read-only question: “Can my monitoring laptop read the agent's uptime?”
Net-SNMP 5.x client example: In an isolated lab, the target is 192.0.2.20. An administrator has already configured a read-only SNMPv3 user named av-read, with authPriv, permitted OIDs and access limited to the monitoring host. Configure matching, supported authentication/privacy algorithms and credentials in the client's protected configuration, rather than putting passwords into a command that can land in shell history. Do not publish that configuration.
snmpget -v 3 -l authPriv -u av-read 192.0.2.20 .1.3.6.1.2.1.1.3.0
The numeric OID avoids needing a locally installed MIB just for this test. Run it twice: the returned uptime should advance. A reset can indicate an SNMP agent restart, not necessarily a whole-device reboot.
A timeout is not a diagnosis. Check address, reachability, UDP access, agent enablement, credentials and permitted source before changing settings. An authorization error can mean the user exists but cannot read that object. Do not use SNMP SET for this exercise.
Sources: Net-SNMP GET tutorial, SNMPv3 options, and client configuration protection.
Discuss a project