Strange behavior when setting cookie value to None in the method-level parameter

[mcpwlk]

After running the following code

s = requests.Session()
s.cookies.update({'from-my': 'browser'})
r = s.get('http://httpbin.org/cookies', cookies={'another': 'cookie', 'from-my': None})
print r.text

the output is

{
  "cookies": {
    "from-my; another": "cookie"
  }
}

I used Python 2.7.6 and requests 2.7.0.

For more information, please see http://stackoverflow.com/questions/31902510/strange-requests-behavior-when-setting-cookie-value-to-none-in-the-method-level

[lukasgraf]

Because I recently updated that section of the docs, I examined this behavior in order to make sure I didn't add incorrect/misleading documentation.

I believe this is because the pattern of "Remove a Value From a Dict Parameter by setting it to None" simply isn't being considered for cookies when merging request cookies with session cookies.

merge_cookies() basically just updates the target jar, but does not remove any items.

This means cookies with a value of None will make it into the RequestsCookieJar, and will be serialized to a Cookie: from-my; another=cookie header.

This is where the parsing the Cookie: header by httpbin.org/cookies also plays a role: Depending on whether the empty cookie appears first or last in the header, it will either combine it into the key of the next cookie, or drop it:

Cookie: from-my; another=cookie  ---->  "cookies": {"from-my; another": "cookie"}
Cookie: another=cookie; from-my  ---->  "cookies": {"another": "cookie"}

So this is why key order seems to matter when testing against httpbin.org/cookies.

Now the question is, should a client even be sending empty cookies like that? RFC 6265 | 5.4. The Cookie Header seems to say no:

  1.  Output the cookie's name, the %x3D ("=") character, and the
       cookie's value.

It doesn't seem to address empty cookie values specifically, but it does require a = delimiter for cookie-pair's. That's also what the grammar for cookie-string states.

Python's cookielib / http.cookiejar seems to think differently:

            if cookie.value is None:
                attrs.append(cookie.name)
            else:
                attrs.append("%s=%s" % (cookie.name, value))

So when None would be used as a sentinel value to mean "Omit this cookie for this request" like for other dict parameters, this would obviously prevent a user from intentionally sending a cookie with an empty value, which currently is possible.

[Lukasa]

Hmm, interesting. I think cookielib just doesn't expect to receive a None value here.

I think we should do two things: firstly, we should ensure that we unset cookies with None values from the method level argument if we can. Secondly, we should consider whether this is a bug report that should be raised upstream. I searched upstream and didn't find any associated bug report, so we might have been the first to hit this.

[sigmavirus24]

So this does seem to be a bug in the standard library. RFC 2965 even shows the =VALUE as being required (even if it didn't use proper ABNF =P). cookielib was implemented at a time when 2965 was the actual standard.

[MikeWinter]

@lukasgraf RFC 6265 addresses the issue of empty cookie values in both the Set-Cookie grammar in section 4.1.1 and its interpretation by the client in section 5.2 wherein nil values are permitted. Therefore a user agent should return an empty value, subject to constraints imposed when the cookie was set (lifetime, path, etc.), in future requests.
As you remark, an equals symbol is required in the relevant headers, which cookielib fails to include.

As a historical note, the grammar in section 3.2.2 of RFC 2965 (and RFC 2901 before it) forbade nil values by specifying cookie values to be tokens (see section 2.2 of RFC 2616) or quoted strings, though the latter could be empty. The original Netscape proposal was silent on the matter.