Lecture 21 Application Programming Interfaces (APIs)

What is it?

A computing interface to a software component or a system, that defines how other components or systems can use it.

API vs. Application

Application/Library/Service
A blob of code that does things
API:
The way your code can ask the application/library/service to do things
The boundary of the application/library through which requests go in and responses come out

APIs vs. UIs

The Browser

API Aspects

Data model
what data does the API manipulate?
Methods
what can you ask it to do?
function names, arguments, return types
Syntax
In what language/format are the method invocations requested?

90s Web: HTTP

HTTP

  • API for talking to Web Servers
  • 6 main methods, pretty much as originally spec'ed by Tim Berners Lee
OPTIONS
“tell me about yourself”
GET
retrieve a resource
POST
upload a new resource
PUT
replace an existing resource
PATCH
modify a resource
DELETE
remove a resource

Method Parameters

Request

  • URL: “this” for method invocation
    • "object" you ask to do things
  • Body: content to PUT/POST/PATCH
  • Headers
    • Referer: page making the request
    • Accept: acceptable response data types (text, pdf, etc.)
    • Cookie: contains specific user data

Response

  • Status code: 200 OK, 301 redirect, 401 Unauthorized, 500 server error…
  • Body: content such as web page
  • Headers:
    • Content-Type: of returned object (text, pdf, etc.)
    • Set-Cookie: user data to store in browser
    • Location: Instruction to look elsewhere

Serialization

  • Network carries a stream of bytes/text
  • Each HTTP request is serialized into a big string with specific syntax
  • Web server receives and deserializes the string back into a structured request
  • Server then serializes response that returns to client
  • Which deserializes it into response data

							POST /echo?urlarg=newUser HTTP/1.1
							Host: scooterlabs.com
							Connection: keep-alive
							Content-Length: 39
							Cache-Control: max-age=0
							Upgrade-Insecure-Requests: 1
							Origin: http://localhost
							Content-Type: application/x-www-form-urlencoded
							User-Agent: Mozilla/5.0 (Windows NT 10.0……
							Accept: text/html,image/webp,img/apng…
							Referer: http://localhost/lectures/apis/form.html
							Accept-Encoding: gzip, deflate
							Accept-Language: en-US,en;q=0.9,he;q=0.8
					

Idempotent and Safe Methods

The Ugly Reality

  • Developers are lazy
  • Can't be bothered with fine semantic distinctions of methods
  • Think only 2 really matter
  • Implement all actions through these

GET

  • arguments in query string (?) of url
  • easy to serialize whole request into url string
  • great for a few short method parameters

POST

  • body can contain serialized arguments
  • need work to build the body correctly
  • necessary for many/big arguments or uploads

HTTP Echo servers

  • Send back whatever you submitted
  • Good for testing your code
  • Examples:
    
    									<a href="http://…/echo?myname=David…">
    
    									<form class="white" method="post"
    									    action="http://…/echo?urlArg=newUser">
    									    Input: <input name="textbox"
    									            type="text">

00s Web: AJAX & REST

Origins

The Popular Approach

  • 90s web applications ran on the server
  • To take an action, user submitted a form
  • To await changes, keep reloading the page
  • Browser just displayed results and accepted user input

The Insight

  • Many actions change little of the UI
  • Client can do significant computation
  • Why bother involving the server?
    • Demands more server horsepower
    • Network makes slow UI
    • Reload loses your place in the page—disruptive
“The classic web application model works like this: Most user actions in the interface trigger an HTTP request back to a web server. The server does some processing — retrieving data, crunching numbers, talking to various legacy systems — and then returns an HTML page to the client. It’s a model adapted from the Web’s original use as a hypertext medium, but as fans of The Elements of User Experience know, what makes the Web good for hypertext doesn’t necessarily make it good for software applications.”

AJAX

Asynchronous
promises etc.
Javascript
computation on client
and XML
data serialization format
now replaced by JSON
but AJAJ doesn't sound as cool

Web APIs

Human vs. Machine

Web

HTML

https://github.com/karger


								<!DOCTYPE html>
								<html lang="en">
								<head>
								<meta charset="utf-8">
								<link rel="dns-prefetch" href="https://github.githubassets.com">
								<link rel="dns-prefetch" href="https://github-cloud.s3.amazonaws.com">
								<link rel="dns-prefetch" href="https://user-images.githubusercontent.com/">

								<link crossorigin="anonymous" media="all" integrity="sha512-bqIvTjrYM8hmo8Y7y5fzsxn+OAFrHWG6byMAbBSM9qFe8NQymQqFeBaF2MGvAwy7x2s4bnzXE5bDGHnrQjtoQQ==" rel="stylesheet" href="https://github.githubassets.com/assets/github-6ea22f4e3ad833c866a3c63bcb97f3b3.css" />

								<meta name="viewport" content="width=device-width">

								<title>karger (David Karger)</title>
								<meta name="description" content="karger has 31 repositories available. Follow their code on GitHub.">
								<link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="GitHub">
								<link rel="fluid-icon" href="https://github.com/fluidicon.png" title="GitHub">
								<meta property="fb:app_id" content="1401488693436528">

								<meta name="twitter:image:src" content="https://avatars3.githubusercontent.com/u/1141086?s=400&v=4" /><meta name="twitter:site" content="@github" /><meta name="twitter:card" content="summary" /><meta name="twitter:title" content="karger - Overview" /><meta name="twitter:description"
								content="karger has 31 repositories available. Follow their code on GitHub." />

								<div class="js-profile-editable-area">
								<div class="hide-sm hide-md">
								<button name="button" type="button" class="btn btn-block mt-2 mb-3 js-profile-editable-edit-button" data-hydro-click="{"event_type":"user_profile.click","payload":{"profile_user_id":1141086,"target":"INLINE_EDIT_BUTTON","user_id":1141086,"originating_url":"https://github.com/karger"}}" data-hydro-click-hmac="15cf48f81af651c181ded3e3432629aaaed4c49127d0ccb02f75963a43bf695d">Edit profile</button>
								</div>

								<div class="p-note user-profile-bio mb-2 js-user-profile-bio"><div>
								Professor of Computer Science at MIT CSAIL, focusing on human computer interaction, end user application development, and computer supported collaborative work.</div></div>

								<ul class="vcard-details mb-3">
								<li itemprop="worksFor" show_title="false" aria-label="Organization: MIT" class="vcard-detail pt-1 css-truncate css-truncate-target"><svg class="octicon octicon-organization" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M16 12.999c0 .439-.45 1-1 1H7.995c-.539 0-.994-.447-.995-.999H1c-.54 0-1-.561-1-1 0-2.634 3-4 3-4s.229-.409 0-1c-.841-.621-1.058-.59-1-3 .058-2.419 1.367-3 2.5-3s2.442.58 2.5 3c.058 2.41-.159 2.379-1 3-.229.59 0 1 0 1s1.549.711 2.42 2.088C9.196 9.369 10 8.999 10 8.999s.229-.409 0-1c-.841-.62-1.058-.59-1-3 .058-2.419 1.367-3 2.5-3s2.437.581 2.495 3c.059 2.41-.158 2.38-1 3-.229.59 0 1 0 1s3.005 1.366 3.005 4z"></path></svg>
								<span class="p-org"><div>MIT</div></span>
								</li>
								<li itemprop="homeLocation" show_title="false" aria-label="Home location: Cambridge, MA" class="vcard-detail pt-1 css-truncate css-truncate-target"><svg class="octicon octicon-location" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M6 0C2.69 0 0 2.5 0 5.5 0 10.02 6 16 6 16s6-5.98 6-10.5C12 2.5 9.31 0 6 0zm0 14.55C4.14 12.52 1 8.44 1 5.5 1 3.02 3.25 1 6 1c1.34 0 2.61.48 3.56 1.36.92.86 1.44 1.97 1.44 3.14 0 2.94-3.14 7.02-5 9.05zM8 5.5c0 1.11-.89 2-2 2-1.11 0-2-.89-2-2 0-1.11.89-2 2-2 1.11 0 2 .89 2 2z"></path></svg>
								<span class="p-label">Cambridge, MA</span>
								</li>
					

API

https://api.github.com
/users/karger


							{
							"login": "karger",
							"id": 1141086,
							"node_id": "MDQ6VXNlcjExNDEwODY=",
							"avatar_url": "https://avatars2.githubusercontent.com/u/1141086?v=4",
							"gravatar_id": "",
							"url": "https://api.github.com/users/karger",
							"html_url": "https://github.com/karger",
							"followers_url": "https://api.github.com/users/karger/followers",
							"following_url": "https://api.github.com/users/karger/following{/other_user}",
							"gists_url": "https://api.github.com/users/karger/gists{/gist_id}",
							"starred_url": "https://api.github.com/users/karger/starred{/owner}{/repo}",
							"subscriptions_url": "https://api.github.com/users/karger/subscriptions",
							"organizations_url": "https://api.github.com/users/karger/orgs",
							"repos_url": "https://api.github.com/users/karger/repos",
							"events_url": "https://api.github.com/users/karger/events{/privacy}",
							"received_events_url": "https://api.github.com/users/karger/received_events",
							"type": "User",
							"site_admin": false,
							"name": "David Karger",
							"company": "MIT",
							"blog": "http://people.csail.mit.edu/karger",
							"location": "Cambridge, MA",
							"email": null,
							"hireable": null,
							"bio": "Professor of Computer Science at MIT CSAIL, focusing on human computer interaction, end user application development, and computer supported collaborative work.",
							"public_repos": 29,
							"public_gists": 1,
							"followers": 100,
							"following": 0,
							"created_at": "2011-10-20T15:41:03Z",
							"updated_at": "2020-04-24T23:02:34Z"
							}
					

Application-Specific APIs

Encapsulation

Wrapper Libraries

RESTful APIs

The Problem

  • In many traditional services, server remembers state of session with client
    • who the user is
    • how far they've gotten in the activity
    • what comes next
  • Lots of this information is ephemeral
    • irrelevant once user closes the page
  • Significant burden on server to maintain
    • server doesn't know user has left
    • must keep state forever for every user who ever visited

The solution

  • REpresentational State Transfer
  • Keep all ephemeral state in the client
  • Every method call must include all the state the server needs to provide the method
  • Server still holds persistent state
    • your file uploads
    • your purchases

Exploration

Chat Client Code Structure

Possible Get Message Code (BAD)


					getMessage(ownAlias, messageId) {
						let route = `messages/${messageId}`;
						let headers = new Headers();
						headers.append("user-alias-name", ownAlias);
						return fetch("https://messaging-server.csail.mit.edu/messages" + messageId,
						{
							method: "GET",
							headers: headers,
						}
						)
						.then(response => {
							if (response.ok) {
								return response;
							} else {
								return response.json().then(err => {
									throw new Error(err.message);
								})
							}
						})
						.then(response => response.json())
						.then(messageDTO => {
							return new Message(
								messageDTO.id,
								new Date(messageDTO.createdAt),
								new Date(messageDTO.updatedAt),
								createAliasFromAliasResponse(messageDTO.sender),
								messageDTO.recipients.map(createAliasFromAliasResponse),
								messageDTO.payload
							);
						});
					}
				

30 lines of code for each request. Most of it will be repeated!

Actual Get Message Code

In Client.js


				  
				    getMessageById(messageId, { handle = this.account.handle } = {}) {
				       return this.#messages.getMessage(handle, messageId);
				    }
				

In MessagesEndpoint.js

                
				  	async getMessage(ownAlias, messageId) {
					  let messageDTO = await this.request(`messages/${messageId}`, {
					       headers: {
					           "user-alias-name": ownAlias,
					       }
					  });

					  return new Message({...messageDTO, data: messageDTO.payload});
	}
				
              

Actual Get Message Code (II)

In util.js


				  export function request (url, {
				      responseType = "json",
				      method = "GET",
				      ...options
				    } = {}) {
				    let fetchOptions = {
				      method,
				      headers: createJSONEncodedHeaders(options.headers),
				      body: options.body ? createJSONEncodedBody(options.body) : null,
				      redirect: "follow",
				      credentials: "include",
				    };

				  return fetch(url, fetchOptions)
				    .then(getErrorFromResponse)
				    .then(response => response[responseType]());
				  }
				  
				

Strategy: Extract common patterns into helper methods and parameterize them.

Create Options for the Request


						function createJSONEncodedHeaders(headers = undefined) {
							let requestHeaders = new Headers();
							if (headers !== undefined) {
								for (let key of Object.keys(headers)) {
									requestHeaders.append(key, headers[key]);
								}
							}
							requestHeaders.append("Content-Type", "application/json");
							return requestHeaders;
						}
					

Boring data transformation; do it once/DRY

Error Handling Helper


					function getErrorFromResponse(response) {
						return new Promise((resolve, reject) => {
							if (response.ok) {
								resolve(response);
							} else {
								reject(response);
							}
						}).catch((responseOrError) => {
							if (responseOrError instanceof Response) {
								return responseOrError.json().then((errorJSON) => {
									console.warn("api error", errorJSON.message);
									throw new Error(errorJSON.message);
								});
							} else {
								throw responseOrError;
							}
						});
					}
				

In Class

'10s: Security & Authorization

Why?

On Clients?

Authorization with Tokens

Web Token Types (I)

In URL parameters
  • http://site/?user=david&pwd=foo
  • terrible idea in general. Why?
    • URLs get saved, copied, shared
    • So tokens leak
    • But ok for new-user invite, password reset
    • Because url can be obsoleted quickly
Cookies
  • Associated with a particular domain
  • Server sends Set-Cookie: header in response from that domain
  • Client sends Cookie: header with (every) request to that domain
  • Consisten with REST:
    • client holds state, always sends to server
  • But remember client is not trusted

Web Token Types (II)

  • HTTP Basic Auth header
    • Authorization: Basic user:password
  • HTTP Digest Auth header
    • Authorization: Digest hash(user:passwd:nonce)
    • hashes username and pwd with nonce for security
  • HTTP Bearer Auth header
    • token is not username/password
    • instead, arbitrary string server can recognize
    • Authorization: bearer r8yc3n8t3ct9
    • decouples identity from authorization
    • server can know you're allowed without knowing who you are

Browser Built-in Handling

  • URL parameters
    • Browser can fetch url with included parameters
  • Cookie
    • Browser stores any cookie sent from server
    • sends cookie to server with every request
  • Bearer authentication
    • Must write custom js code to handle
  • Basic and Digest Auth
    • Server responds with challenge:
      
      									HTTP/1.1 401 Unauthorized
      									Date: Wed, 21 Oct 2015 07:28:00 GMT
      									WWW-Authenticate: Basic realm="asgard"
      							
    • Browser opens user/passwd dialog
    • and puts answer in appropriate Authentication: header
    • cached so user needn't re-type
    • Try it here

Handling in JS Code

  • URL
    • fetch()
  • Cookie
    • document.cookie property to set/get cookie
    • browser stores, sends with every request
  • Basic/Digest/Bearer Auth
    • Observe server challenge in response object
    • Send relevant Auth headers in fetch
      
      										fetch("https://httpbin.org/basic-auth/user/passwd", {
      										headers: new Headers({
      										"Authorization":
      										    `Basic ${base64.encode(`${user}:${pwd}`)}`
      										}),
      										}).then(response => {
      										if (!response.ok) throw new Error(response.status);
      										return response.json(); })
      								

Acquiring Tokens

Your Messaging Library uses Cookies


					function login(email, password) {
						let route = "login";
						return fetch(
						`${this.store.host}/${route}`,
						createDefaultRequestInit({
							method: "POST",
							body: { email, password }
						})
						)
						.then(getErrorFromResponse)
						.then((res) => res.json())
						.then((accountDTO) => {
							return createAccountFromAccountDTO(accountDTO);
						});
					}
				
Login causes the server to send back a cookie

					function logout() {
						let route = "logout";
						return fetch(
							`${this.store.host}/${route}`,
							createDefaultRequestInit({ method: "POST" })
						)
						.then(getErrorFromResponse)
						.then((res) => {
							return;
						});
					}
				
Logout causes the server to send a null cookie.

API Keys

Security

Eavesdropper Attacks

Man In The Middle

Same Origin Policy

Some Attacks & Defenses

Stealing cookies/local storage
  • Attack:
    • Copy cookie from other page?
  • Defense:
    • Code like document.cookie only accesses cookie for domain of own page.
Iframes
  • Attack:
    • load target bank in iframe
    • iframe.contentDocument gives access to DOM inside iframe
    • inspect contents to steal data
    • submit withdrawal request
  • Defense:
    • Same origin policy means access to other-origin iframe.contentDocument returns null

Cross Site Scripting (XSS)

Attack

  • General attack: get your malicious script to run on (page from) target domain
  • Running in that domain, it has access to that domain's content
    • Can read/modify cookies
    • Can modify content on page to trick user
    • Can make API calls using domain's tokens

Example: Script Injection

  • Messaging client may display messages by appending to DOM with elt.innerHTML
  • Send a message:
    Hi Bob,<script src="http://my/my-attack.js"></script> how are you?
  • When message is appended, script runs in domain of app
  • Steal credentials of messaging client
  • Impersonate recipient

Cross Site Scripting

Example

  • Messaging client displays messages by appending to DOM with elt.innerHTML
  • Send a message:
    Hi Bob,<script src="http://my/my-attack.js"></script> how are you?
  • When message is appended, script runs in domain of app
  • Steal credentials of messaging client
  • Impersonate recipient

Defense

  • sanitize any content before you display it
  • make sure it isn't interpreted in an executable way
    • e.g., replace < character with &lt; entity
    • elt.textContent does this
  • same caution r.e. content you store on your server

What about script tags?

Question

  • A page can include <script> tags linking to other sites
  • This is important; it lets us link to useful remote libraries
  • JS can even dynamically inject such script tags in the page

Answer

  • Those scripts run in my domain
    • Cannot access other site!
  • As if I copied that code to my app
  • They're in my app because I put them there
  • Browser assumes I know what I'm doing
  • But note risk: malicious library can change its code later!

Cross-Site API Calls

  • Sometimes you want to pull data from other sites into your page
  • Or allow interaction with those sites, from your page
  • Iframe?
    • Style mismatch with your page
    • May want to combine that data with yours
    • May want to run that site's methods on your data
Embedded tweet pulling live data from Twitter

Should this be allowed?

No

  • If my company built an API, often I want only my site to have access
    • Don't want competitor sites stealing functionality of my API on their site!
    • Could require login to access, but
      • Requires users to setup accounts, which I might not want
      • Competitor could still steal my customers by using my API for them

Yes

  • I might want to share my API widely
    • Social good
    • Advertising/attractor for my site
  • I might want to share my API narrowly
    • Allow access by a few partner services

Cross Origin Resource Sharing (CORS)

OAuth

Motivation

  • Big hassle to have to log in to every site separately
  • Once the browser's knows I'm me, can't it prove it to every site where I have an account?
  • Skip all those per-site passwords, just prove I'm me once?
  • Some kind of globally accepted token?

Approach

  • Problem: if one token is globally accepted, everyone has it
    • So they can all pretend to be me?
  • Solution:
    • Distinct token per site
    • OAuth server generates tokens for me
    • I only need one permanent token
    • Use it to instruct OAuth server to create tokens for other sites

OAuth providers (servers)

OAuth providers (servers)

Summary: Web APIs

  • HTTP
    • 6 verbs GET/POST/PUT&hellp;
    • Headers carry method parameters
  • Web Application APIs
    • JSON object serialization
    • fetch() method
    • RESTFUL applications—all state on client


  • Please go/feedback
  • Authentication
    • Token to prove you are allowed
    • Might identify you or not
    • Different types
      • Params, Cookie, Basic, Digest, Bearer
    • OAuth offers centralization
  • Security
    • https vs. eavesdroping, man-in-middle
    • Same-Origin Policy protects sites
      • Script injection attacks try to hack
      • CORS offers controlled relaxation