Good Day,
I have a problem for any CM ( of diferents vendors and diferents models) when i try upgrade the firmware, the CM start download the image for upgrade but after some minutes the download fail and the conection TFTP closed, this only happen in BSR64K and with some CM no all (maybe 1500 of 14000). the TFTP server have config the timeout in "35 mins" and maxrequest in "2".
CM response:
[CmDocsisIpThread] BcmNonVolDeviceDriverBridge::WriteSync: (NonVol Device) Synchronous write to dynamic nonvol section succeeded
CheckLinuxState : linuxBootState=0
Trying to create a buffer of 65536 bytes (64k) to store the image...
Done! Actually allocated 65536 bytes.
Opening file 'firmware.p7b' on XXX.XX.XXX.XXX for reading...
CM_Console> [12:37:33 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) TFTP blocksize value returned by server: 1448
Loading from server...
CM_Console>
Loaded 66759 bytes.
Monolithic image detected...
ProgramStore header was verified. Image can be downloaded.
[12:37:33 02/19/2019] [CmDocsisIpThread] BcmProgramStoreDeviceDriverBridge::Open: (Program Store Device)
Opening the default image number (1).
NandFlashRead: Detected out-of-order block @offset 0x4c40200, tagged offset 0xffffff00, expected offset 0x200
NandFlashRead: Failed to find replacement block!
Storing to device...
Stored 63488 bytes.
Loading from server...
CM_Console>
Loaded 65536 bytes.
Storing to device...
Stored 129024 bytes.
Loading from server...
CM_Console>
Loaded 131072 bytes.
Storing to device...
Stored 194560 bytes.
Loading from server...
CM_Console>
Loaded 196608 bytes.
Storing to device...
Stored 260096 bytes.
Loading from server...
CM_Console>
Loaded 262144 bytes.
Storing to device...
Stored 325632 bytes.
Loading from server...
CM_Console>
Loaded 327680 bytes.
Storing to device...
Stored 391168 bytes.
Loading from server...
CM_Console>
Loaded 393216 bytes.
Storing to device...
Stored 456704 bytes.
Loading from server...
CM_Console>
Loaded 458752 bytes.
Storing to device...
Stored 522240 bytes.
Loading from server...
CM_Console>
Loaded 524288 bytes.
Storing to device...
Stored 587776 bytes.
Loading from server...
CM_Console>
Loaded 589824 bytes.
Storing to device...
Stored 653312 bytes.
Loading from server...
CM_Console>
Loaded 655360 bytes.
Storing to device...
Stored 718848 bytes.
Loading from server...
CM_Console>
Loaded 720896 bytes.
Storing to device...
Stored 784384 bytes.
Loading from server...
CM_Console>
Loaded 786432 bytes.
Storing to device...
Stored 849920 bytes.
Loading from server...
CM_Console>
Loaded 851968 bytes.
Storing to device...
Stored 915456 bytes.
Loading from server...
CM_Console>
Loaded 917504 bytes.
Storing to device...
Stored 980992 bytes.
Loading from server...
CM_Console> [12:37:45 02/19/2019] [CmDocsisIpThread] Tftp Client::GetReply: (Tftp Client) Timed out on socket select!
[12:37:45 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) Attempt #(1) Backoff (1) Exp Block #(689) Last Block #(688) Recv'd Block #(688)
CM_Console> [12:37:47 02/19/2019] [CmDocsisIpThread] Tftp Client::GetReply: (Tftp Client) Timed out on socket select!
[12:37:47 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) Attempt #(2) Backoff (2) Exp Block #(689) Last Block #(688) Recv'd Block #(688)
CM_Console> [12:37:51 02/19/2019] [CmDocsisIpThread] Tftp Client::GetReply: (Tftp Client) Timed out on socket select!
[12:37:51 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) Attempt #(3) Backoff (4) Exp Block #(689) Last Block #(688) Recv'd Block #(688)
CM_Console> [12:37:59 02/19/2019] [CmDocsisIpThread] Tftp Client::GetReply: (Tftp Client) Timed out on socket select!
[12:37:59 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) Attempt #(4) Backoff (8) Exp Block #(689) Last Block #(688) Recv'd Block #(688)
CM_Console> [12:38:15 02/19/2019] [CmDocsisIpThread] Tftp Client::GetReply: (Tftp Client) Timed out on socket select!
[12:38:15 02/19/2019] [CmDocsisIpThread] Tftp Client::Send: (Tftp Client) ERROR - Request Retries > kTftpMaxRequestRetryCount (4) !
Loaded 929465 bytes.
[12:38:15 02/19/2019] [CmDocsisIpThread] BcmCmDocsisIpThread::TftpMonolithicImage: (CmDocsisIpThread) ERROR -
Failed to read a buffer from the server!
TftpMonolithicImage download unsuccessful.
[12:38:15 02/19/2019] [CmDocsisIpThread] BcmProgramStoreDeviceDriverBridge::Close: (Program Store Device) WARNING - There was a problem with loading or storing the image; it will not be locked in!
[12:38:15 02/19/2019] [CmDocsisIpThread] BcmNonVolDeviceDriverBridge::WriteSync: (NonVol Device) Writing to dynamic nonvol section
Logging event: SW upgrade Failed before download -TFTP Max Retry Exceeded
Logging event: SW upgrade Failed before download -TFTP Max Retry Exceeded
[12:38:15 02/19/2019] [CmDocsisIpThread] BcmNonVolDeviceDriverBridge::WriteSync: (NonVol Device) Synchronous write to dynamic nonvol section succeeded
[12:38:15 02/19/2019] [CmDocsisIpThread] BcmProgramStoreDeviceDriverBridge::ClearImage: (Program Store Device)
Clearing image number 3.
NandFlashRead: Detected out-of-order block @offset 0x6740200, tagged offset 0xffffff00, expected offset 0x200
NandFlashRead: Failed to find replacement block!
remove kernel image anyway to force a Firmware upgrade on next reboot!!
Monolithic download returns...
Hi,
Just a though: did you see this problem only on D2.0 or D3.0 CMs?
If you see the problem only on D3.0 CM, have you checked the downstream signal quality? Maybe you have some D3.0 running with downstream bonding but some downstreams has to much errors but didn't move into partial service. Then if your CM is receiving a firmware upgrade via TFTP which is using UDP some packets can go lost due the resequencing over downstreams with high errors.
For this case I propose to use downstream/upstream service flows without bonding capabilities (forbidden attribute mask 0x80000000) and IP Classifier for the network where the TFTP-Server is located. This service flow will normally use the primary downstream channel and this channel is in the most time good enough. Otherwise the D3.0 CM will drop off anyway if the primary downstream channel has to much errors.
On newer D3.0 and D3.1 CM I recommend FW-Uprades over HTTP anyway.
Best regards