Make an HTTP(S) GET request and read the response using the Windows API

This post is one of a series providing implementation examples of Windows API Functions, Types, Enums and Consts using VBA. The code in this post can be used as-is, however, if you regularly (or even just occasionally) work with Windows API declarations in VBA, you may want to see the posts Automatically add Windows API declaration(s) and Using 'F1' to view Windows API web pages which explain some of the functionality that can be added to the VBE by VBE_Extras.

Sooner or later, as a VBA developer, you're going to need to get information from the web ... to fetch a page, call a REST endpoint, or read a small text or JSON feed. The usual answer is a COM object such as MSXML2.XMLHTTP or WinHttp.WinHttpRequest, which means you are relying on the presence of the relevant library(s) on your (or an end-users) device. There's nothing wrong with that so long as the library(s) are present! However, Windows already has an HTTP stack built in, WinINet, and a handful of its Functions will do a GET for you with no library required.

A few things to know before the code:

• A WinINet GET is a short sequence of steps, and each one hands you an HINTERNET handle that you must later close with InternetCloseHandle: InternetOpen starts a session, InternetConnect opens a connection to the host, HttpOpenRequest creates the GET, and HttpSendRequest sends it. Once it's gone, HttpQueryInfo reads the status code and InternetReadFile reads the body.

• You pass the host and the path separately ... "example.com" and "/", not one combined URL.

• The following code uses the "W" (Unicode) variants of the Functions, so every String parameter is declared As LongPtr and passed using StrPtr (see the relevant paragraph in Things to note ... for details on why)

• HTTPS (as opposed to HTTP) needs no extra effort: it's just the INTERNET_FLAG_SECURE flag and port 443. Windows does the TLS handshake for you.

The code

Add a standard Module and paste in all of the following.

First the declarations ... the Consts, and Function declarations in their 32- and 64-bit forms:

Next, the Sub you run to test the code ... it does a GET of example.com over HTTPS, then prints the status code and the first part of the response to the Immediate window:

The Function that does the work ... it walks the session > connection > request > send sequence, then reads the status code and body, closing every handle on the way back out:

And two helpers ... one reads the numeric status code, the other reads the whole body in 8 KB chunks and builds it into a String:

Things to note ...

Reaching the server is not the same as success. HttpGet returns True when the request completed and a response came back ... but that response might still be a failure code. The HTTP status code is a separate thing, which is why it comes back in its own argument ... so check outlStatusCode for 200 before you trust the body.

The response comes back as bytes. InternetReadFile gives you raw bytes, so ReadResponseBody reads them into a Byte array and turns each byte into a character with StrConv(..., vbUnicode). That's exactly right for plain ASCII or Latin-1 text (which covers most simple pages and many APIs), but it is not a UTF-8 decoder ... if you're pulling back UTF-8 with accented or non-Latin characters, you'd want to decode the bytes properly. As so much of what you read from the web is UTF-8, there's a drop-in UTF-8 version of ReadResponseBody in the Reading the response as UTF-8 section, below.

The "W" Functions and StrPtr. A VBA String is already UTF-16 internally, which is exactly what the "W" (wide, i.e. Unicode) Functions expect. So, rather than declare the String parameters As String and let VBA quietly convert them to ANSI (which is the default VBA behaviour), the declarations take each one As LongPtr and we hand over StrPtr(theString) ... the address of the String's own buffer. That keeps everything Unicode from end-to-end (there are ANSI "A" versions of these Functions too but using them would throw that Unicode away for no benefit).

No Reference. wininet.dll ships with Windows, so there is nothing to add under Tools > References and nothing to install.

Reading the response as UTF-8

StrConv(..., vbUnicode) treats each byte as one character, which is fine for ASCII or Latin-1 ... but a great deal of what you'll read from the web is UTF-8, where a single accented or non-Latin character is made up of two, three or four bytes. Decoded one byte at a time, those come out as mojibake (é where you wanted é, and so on).

The fix is to hand the raw bytes to Windows and let it do the UTF-8 decoding, using the MultiByteToWideChar Function. There are two things to remember when doing this:

• Keep all the response bytes and decode them only once, at the end (a multi-byte UTF-8 character can straddle the boundary between two 8 KB chunks, so decoding chunk-by-chunk would corrupt those characters)

• Call the MultiByteToWideChar Function twice ... once to find out how many characters the result needs, then again to fill the String.

To switch to the UTF-8 version, add these declarations alongside the others at the top of the Module:

... and then replace the earlier ReadResponseBody with these two Functions ... the first accumulates the raw bytes and hands them to the second, which decodes them in one go:

With that in place, HttpGet behaves exactly as before ... it just returns a correctly decoded String when the server sends UTF-8.

Finally ...

A session, a connection, a request ... four calls to send a GET and two more to read what came back, with no external library(s) to depend on. From there it's a small step to a POST, to adding your own request headers, or to handing the response off to a JSON or text parser.