Voting resources, early voting, and poll worker information - VOTE. ... Adafruit is open and shipping.
0

wget didn't write a complete file
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

wget didn't write a complete file

by slowjim on Thu Feb 13, 2020 9:27 pm

Hi,

I am trying to get my PyPortal Titano set up with the NASA Apod. I have come across a few hitches along the way which I have overcome, but I am now stuck. When the code loads up I get the following REPL feedback:

code.py output:
Set background to /pyportal_startup.bmp
ESP firmware: bytearray(b'1.2.2\x00')
Connecting to AP XXXXXXXXXX
Set background to /nasa_background.bmp
Retrieving data...Reply is OK!
original URL: https://apod.nasa.gov/apod/image/2002/t ... rR1024.jpg
convert URL: https://io.adafruit.com/api/v2/slowjim/ ... rR1024.jpg
Fetching stream from https://io.adafruit.com/api/v2/slowjim/ ... rR1024.jpg
Didn't receive full response, failing out
Some error occured, retrying! - wget didn't write a complete file
Retrieving data...Reply is OK!
original URL: https://apod.nasa.gov/apod/image/2002/t ... rR1024.jpg
convert URL: https://io.adafruit.com/api/v2/slowjim/ ... rR1024.jpg
Fetching stream from https://io.adafruit.com/api/v2/slowjim/ ... rR1024.jpg
Traceback (most recent call last):
File "code.py", line 37, in <module>
File "code.py", line 34, in <module>
File "adafruit_pyportal.py", line 913, in fetch
File "adafruit_pyportal.py", line 910, in fetch
File "adafruit_pyportal.py", line 896, in fetch
File "adafruit_pyportal.py", line 890, in fetch
File "adafruit_pyportal.py", line 666, in wget
KeyError: content-length

So it looks like it successfully connects to the wifi, identifies the web page but then comes throws an error: Didn't receive full response, failing out, Some error occured, retrying! - wget didn't write a complete file. It tries again and then throws the final error.

I have put a copy of my code below (note the only things I have changes are I added some libraries as per advice and I changed the sleep at the bottom to 10 seconds so I could see what was happening in the code) :

Code: Select all | TOGGLE FULL SIZE
import time
import board
from adafruit_pyportal import PyPortal
import busio
from digitalio import DigitalInOut
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
import adafruit_requests as requests

# Set up where we'll be fetching data from
DATA_SOURCE = "https://api.nasa.gov/planetary/apod?api_key=TEFHvH0LHbXXXXXXXX6gypz98lxasavBz6HUhe"
# There's a few different places we look for data in the photo of the day
IMAGE_LOCATION = ["url"]
TITLE_LOCATION = ["title"]
DATE_LOCATION = ["date"]

# the current working directory (where this file is)
cwd = ("/"+__file__).rsplit('/', 1)[0]
pyportal = PyPortal(url=DATA_SOURCE,
                    json_path=(TITLE_LOCATION, DATE_LOCATION),
                    status_neopixel=board.NEOPIXEL,
                    default_bg=cwd+"/nasa_background.bmp",
                    text_font=cwd+"/fonts/Arial-12.bdf",
                    text_position=((5, 220), (5, 200)),
                    text_color=(0xFFFFFF, 0xFFFFFF),
                    text_maxlen=(50, 50), # cut off characters
                    image_json_path=IMAGE_LOCATION,
                    image_resize=(320, 240),
                    image_position=(0, 0))

while True:
    response = None
    try:
        response = pyportal.fetch()
        print("Response is", response)
    except RuntimeError as e:
        print("Some error occured, retrying! -", e)

    time.sleep(10)  # 30 minutes till next check

slowjim
 
Posts: 37
Joined: Fri Sep 27, 2019 3:49 am

Re: wget didn't write a complete file

by cyberstar on Sat Jun 20, 2020 2:16 pm

I've been having similar issues with wget failing and the NASA picture of the day not working. Both the "content-length" error and the "wget didn't write a complete file" error.

I'm running on a DSL ~5Mbps WAN line and experiencing various wget method errors within the PyPortal object.
Tracing through the PyPortal code library from 6/16 I found on line 717 " r = requests.get(url, stream=True)" that there was no "timeout" parameter passed into the requests.get call. Changing the line to read " r = requests.get(url, stream=True, timeout=10)" resolved all of the errors I was seeing.

The default "timeout" parameter in the "requests.get" method is defaulted to "1" which is clearly to short for slower WAN / internet connections.

Change the "requests.get" method def to use a higher timeout default say "10".
Or you can change the PyPortal call to requests.get within "wget" method to pass in an appropriate timeout parameter value.

I'm thinking an even better library enhancement is to modify the PyPortal def to add a new timeout parameter thus allowing users to set their own timeout value should it be required and add the "timeout" parameter to the "requests.get" call at line #717.

I'll have to go back and look into the Youtube fix posted elsewhere in the forum and git hub as that outlier popped up on me as well but the solution offered was failing due to the above mentioned wget timeout issue.

Hope that helps.

cyberstar
 
Posts: 1
Joined: Thu Jun 11, 2020 4:48 pm

Re: wget didn't write a complete file

by siddacious on Mon Jun 22, 2020 8:11 pm

Hi both, the Adafruit_CircuitPython_PyPortal library was updated to lengthen the default timeout to 10 seconds. Please make sure you have the latest version of the library bundle for the version of CircuitPython installed on your PyPortal: https://circuitpython.org/libraries

siddacious
 
Posts: 310
Joined: Fri Apr 21, 2017 3:09 pm

Re: wget didn't write a complete file

by gmeader on Mon Aug 10, 2020 4:01 pm

I am having similar problems with reliable PyPortal Titano Internet connectivity:
(I have tested this on various WIFI networks where ordinary browsing works fast and fine.)

PyPortal Titano: WiFi requests are often failing.
Running esp32spi_simpletest.py
ESP Firmware vers. bytearray(b'1.6.1\x00')
adafruit-circuitpython-bundle-5.x-mpy-20200810

Code: Select all | TOGGLE FULL SIZE
Ping google.com: 20 ms
Fetching text from http://wifitest.adafruit.com/testwifi/index.html
Traceback (most recent call last):
  File "code.py", line 74, in <module>
  File "adafruit_requests.py", line 291, in get
  File "adafruit_requests.py", line 251, in request
  File "adafruit_requests.py", line 235, in request
  File "adafruit_esp32spi/adafruit_esp32spi_socket.py", line 102, in readline
RuntimeError: Didn't receive full response, failing out


Often this works, but sometimes this basic example fails.

Worse yet:
In addition the Pyportal library that many of the Adafruit Guides are based on is unreliable.
The code for “wget” in the PyPortal library seems to be where errors occur..
Multiple Adafruit Guides examples that fetch images such as NASA, CMA Art Frame, and NewNewNew
that are using the PyPortal library fail most of the time (always?) on a PyPortal Titano with an SDcard.
I have not yet modified any of the code from these Guides to support the larger screen on the Titano.
This is the stock code from the Guides as downloaded in the zip file

I have never gotten the NASA or NewNewNew to ever complete downloading an image.
The CMA Art code displayed an image twice within 50 retrys.
The API fetches of JSON data such as the image URL usually work, but the image download fails.

Versions installed are most recent:
ESP Firmware vers. bytearray(b'1.6.1\x00')
adafruit-circuitpython-bundle-5.x-mpy-20200810

Errors include:
wget didn’t write a complete filename
keyError: content-length
timed out waiting for SPI char

gmeader
 
Posts: 16
Joined: Sun Dec 06, 2015 6:25 pm

Re: wget didn't write a complete file

by kevinjwalters on Tue Aug 11, 2020 6:53 am

"timed out waiting for SPI char" might be the key error here - there is a ticket that mentions this GitHub: adafruit/Adafruit_CircuitPython_ESP32SPI: No recovery after Timed out waiting for SPI char #102.

kevinjwalters
 
Posts: 742
Joined: Sun Oct 01, 2017 3:15 pm

Re: wget didn't write a complete file

by gmeader on Thu Aug 13, 2020 10:15 am

I see that @tannewt is rewriting the ESP internet connectivity code.
I’d like to request that testing/examples include a program that downloads a few large binary files (larger than a few megabytes) such as images, zip files and databases and stores them on the SDcard. I have a 32GB SDcard on my PyPortal that I need to fill up...
It would be nice to have the download timing and/or progress reported too.

gmeader
 
Posts: 16
Joined: Sun Dec 06, 2015 6:25 pm

Please be positive and constructive with your questions and comments.