Concepts of Programming Languages

Closures

Instructor: Stefan Mitsch

What Problem do Closures Solve?

How to create a "container" for data and functions?

                
                  public class Incrementor {
                    private int i;
                    public Incrementor(int i) { 
                      this.i = i; 
                    }
                    public int increment(int x) {
                      return x+i;
                    }
                  }
                  // use object
                  Incrementor inc = new Incrementor(2);
                  inc.increment(4); // returns 6
                  inc.increment(5); // returns 7
                
              

                
                  def incrementor(i:Int) : Int=>Int = {
                    def increment(x:Int) = x+i
                    return increment;
                  }
                  // use closure
                  val inc = incrementor(2) 
                  inc(4) // returns 6
                  inc(5) // returns 7
                
              

What are the challenges of making closures work?

Closures

• Runtime support for nested functions
  • particularly when lifetimes do not nest
• Only applies to static / lexical scope

Top-Level Functions

• Function declarations made at top level
• Not hidden by scope

int loop (int n, int result) {
  if (n <= 1) {
    return result;
  } else {
    return loop (n - 1, n * result);
  }
}
int fact (int n) {
  return loop (n, 1);
}
          

Nested Functions - GCC

• Nested functions allow for reuse of inner function name
• Allowed by GCC, but not C standard

int fact (int n) {
  int loop (int n, int result) {
    if (n <= 1) {
      return result;
    } else {
      return loop (n - 1, n * result);
    }
  }
  return loop (n, 1);
}
          

$ gcc -c nested-fact.c 
$ gcc -pedantic -c nested-fact.c 
function.c: In function ‘fact’:
function.c:2:3: warning: ISO C forbids nested functions [-pedantic]
          

Nested Functions - GCC

• Access variables from enclosing context
  • requires some runtime support

int fact (int n) {
  int loop (int i, int result) {
    if (i > n) {
      return result;
    } else {
      return loop (i+1, i * result);
    }
  }
  return loop (1, 1);
}
          

Nested functions: Scoping

• Access variable x from which context?
  • requires some runtime support

            
              typedef int (*funcptr) (int);
              {
                int x = 4;
                {
                  int f(int y) { return x*y; }
                  {
                    int g(funcptr h) {
                      int x = 7;
                      return h(3) + x;
                    }
                    g(f)
                  }
                }
              }
            
          

Nested Functions in Scala

def mapDebug [A,B] (xs:List[A], f:A=>B) : List[B] = 
  def printElt (x:A) : B = 
    println (x)
    f (x)  // use f from enclosing context
  xs.map (printElt)
          

def mapDebug [A,B] (xs:List[A], f:A=>B) : List[B] = 
  xs.map ((x:A) => { println (x); f (x) })  // anonymous function clearer
          

Nested Functions - Scope vs Lifetime

• Limit scope of inner function
• Lifetime of inner function vs lifetime of outer function?
• Potentially unsafe, and requires more runtime support
  • more support than accessing variables from enclosing function

Nested Functions - GCC

• From GCC Nested Functions (GCC extensions)

• Lifetime problems!
• Lexical Closures for C++ (T. M. Breuelm, 1988)

Nested Functions - GCC

• Lifetime problems caused by nested functions

typedef void (*funcptr) (int);

funcptr f (int x) {
  void g (int y) {
    printf ("x = %d, y = %d\n", x, y);
  }
  g (1);
  return &g;
}
int main (void) {
  funcptr h = f (10);
  (*h) (2);
  f (20);
  (*h) (3);
}

Nested Functions - GCC

• Unsafe calls may or may not work

$ gcc -std=c99 nested-gcc.c
$ ./a.out
x = 10, y = 1 <- safe to call g, with x=10
x = 10, y = 2 <- unsafe to call h, created with x=10, GOOD!
x = 20, y = 1 <- safe to call g
x = 20, y = 3 <- unsafe to call h, created with x=10, BAD!
          

Nested Function - Clang

• Clang and LLVM
• Apple's Blocks extension to C = nested functions
  • Ars Technica - Snow Leopard review (2009)
  • Apple Developer Library: Introduction to Blocks
• Applications (iOS and OSX)
  • Graphical user interface callbacks
  • Collections processing
  • Concurrent tasks

Nested Function - Clang

#include <Block.h>
// ^funcptr for blocks; *funcptr for function pointers
typedef void (^funcptr) (int);
funcptr f (int x) {
  funcptr g;
  g = ^(int y) {
    printf ("x = %d, y = %d\n", x, y); // use x from enclosing defn
  };
  g = Block_copy (g);
  g (1); // OK, f's activation record still allocated
  return g;
}
int main (void) {
  funcptr h = f (10);
  h (2); // OK, because of Block_copy
  f (20);
  h (3); // OK, because of Block_copy
  Block_release	(h);
}
          

Nested Function - Clang

• Blocks support is not turned on by default on Ubuntu

$ sudo apt-get install libblocksruntime-dev
          

• Correct output

$ clang -fblocks nested-clang.c -lBlocksRuntime
$ ./a.out
x = 10, y = 1
x = 10, y = 2
x = 20, y = 1
x = 10, y = 3 <- safe to call h, created with x=10, GOOD!
          

Nested Function - Clang

• Without Block_copy and Block_release

$ clang -fblocks nested-clang.c -lBlocksRuntime
$ ./a.out 
x = 10, y = 1
x = -1035955720, y = 2  <- unsafe to call h, created with x=10, BAD!
x = 20, y = 1
x = -1035955720, y = 3  <- unsafe to call h, created with x=10, BAD!
          

Nested Function - Scala

• Nested functions work correctly in Scala

def f (x:Int) : Int=>Unit =
  def g (y:Int) : Unit = println ("x = %d, y = %d".format (x, y))
  g (1)
  g

def main () = 
  val h = f (10)
  h (2)
  f (20)
  h (3)

main()
          

x = 10, y = 1
x = 10, y = 2
x = 20, y = 1
x = 10, y = 3 <- safe to call h, created with x=10, GOOD!
          

Nested Function - Scala

• Alternative definition of inner function
• Technical distinction between Scala functions and methods

def f (x:Int) : Int=>Unit = 
  val g : Int => Unit = (y) => println ("x = %d, y = %d".format (x, y))
  g (1)
  g
          

Nested Function - Java 8

• Nested functions work correctly in Java 8

import java.util.function.IntConsumer;

static IntConsumer f (int x) {
  IntConsumer g = y -> System.out.format ("x = %d, y = %d%n", x, y);
  g.accept (1);
  return g;
}
public static void main (String[] args) {
  IntConsumer h = f (10);
  h.accept (2);
  f (20); 
  h.accept (3);
}
          

x = 10, y = 1
x = 10, y = 2
x = 20, y = 1
x = 10, y = 3 <- safe to call h, created with x=10, GOOD!
          

Nested Function - Java 8

• With explicit types

import java.util.function.Function;
static Function<Integer,Void> f (int x) {
  Function<Integer,Void> g = y -> { 
    System.out.format ("x = %d, y = %d%n", x, y); 
    return null;
  };
  g.apply (1);
  return g;
}
public static void main (String[] args) {
  Function<Integer,Void> h = f (10);
  h.apply (2);
  f (20); 
  h.apply (3);
}
          

Nested Function - Java 8

• With explicit object instantiation

import java.util.function.Function;
static Function<Integer,Void> f (int x) {
  Function<Integer,Void> g = new Function<Integer,Void>() {
    public Void apply(Integer y) {
      System.out.format ("x = %d, y = %d%n", x, y); 
      return null;
    }
  };
  g.apply (1);
  return g;
}
public static void main (String[] args) {
  Function<Integer,Void> h = f (10);
  h.apply (2);
  f (20); 
  h.apply (3);
}
          

Problem Summary

1. enclosing function outer is called
   • AR contains data x
2. outer returns nested function inner
   • inner references x from outer's AR
   • lifetime of outer's AR and x ends
3. nested function inner is called
   • needs x from outer's AR

def outer (x:A) : B=>C = 
  def inner (y:B) : C = 
    //...use x and y...
  inner
          

Implementation: Closures

• Closures store inner function and environment
• Environment contains variables from enclosing scope
• Lifetime of environment = lifetime of inner function
  • environment is allocated on the heap
• Different implementations in different PLs
• Recurring implementation choice: copy or share?

Implementation Choice

• Closure contains
  • pointer/reference to code for inner
  • a copy of x

def outer (x:A) : B=>C = 
  def inner (y:B) : C = 
    ...use x and y...
  inner
          

Implementation Choice

• Closure contains
  • pointer/reference to code for inner
  • copies of x and u
• inner sees updated u?
• Require u to be immutable?

def outer (x:A) : B=>C = 
  var u:A = x
  def inner (y:B) : C = 
    //...use u and y...
  u = u + 1
  inner
          

Implementation Choice

• Alternatively, share u
• Closure contains
  • pointer/reference to code for inner
  • copy of x
  • reference to shared u (on heap)

def outer (x:A) : B=>C = 
  var u:A = x
  def inner (y:B) : C = 
    //...use u and y...
  u = u + 1
  inner
          

Scala Implementation

object Closure:
  def outer (x:Int) : Boolean=>Int = 
    def inner (y:Boolean) : Int = 
      x + (if y then 0 else 1)
    inner
          

$ scalac Closure.scala

$ ls -1 Closure*
Closure$$anonfun$outer$1.class
Closure.class
Closure$.class
Closure.scala
          

Scala Implementation

• The closure is an instance of the second class
• x copied into field x$1

$ javap -p Closure
Compiled from "Closure.scala"
public final class Closure {
  public static scala.Function1<java.lang.Object, java.lang.Object> outer(int);
}

$ javap -p Closure\$\$anonfun\$outer\$1
Compiled from "Closure.scala"
public final class Closure$$anonfun$outer$1 extends scala.runtime.AbstractFunction1<java.lang.Object, java.lang.Object> {
  private final int x$1;
  public final int apply(boolean);
  public Closure$$anonfun$outer$1(int);
}
          

Scala Implementation

• u is a var declaration, so is mutable

object Closure:
  def outer (x:Int) : Boolean=>Int =
    var u:Int = x
    def inner (y:Boolean) : Int =
      x + u + (if y then 0 else 1)
    inner
          

Scala Implementation

• x copied into field x$1
• u shared on heap via reference in field u$1

$ javap -p Closure\$\$anonfun\$outer\$1
Compiled from "Closure.scala"
public final class Closure$$anonfun$outer$1 extends scala.runtime.AbstractFunction1<java.lang.Object, java.lang.Object> {
  private final int x$1;
  private final scala.runtime.IntRef u$1;
  public final int apply(boolean);
  public Closure$$anonfun$outer$1(int, scala.runtime.IntRef);
}
          

Another example

val f:()=>Int = {var x= -1; () => { x = x + 1; x}}
val g:Int=>()=>Int = (y) => {var z= y; () => { z = z + 1; z}}
          

scala> f()
res0: Int = 0
scala> f()
res1: Int = 1
scala> val h1=g(10)
h1: () => Int = $$Lambda$1098/39661414@54d8c20d
scala> val h2=g(20)
h2: () => Int = $$Lambda$1098/39661414@5bc7e78e
scala> h1()
res3: Int = 11
scala> h1()
res4: Int = 12
scala> h2()
res5: Int = 21
scala> h2()
res6: Int = 22
scala> h1()
res7: Int = 13
scala> h2()
res8: Int = 23          

Further reading

• There are some great resources about closures in the javascript world, such as this