Concepts of Programming Languages

Scope and Lifetime

Instructor: Stefan Mitsch

Scope

  • Scope of an identifier
    • region of text in which it may be used

Common Scope Rules

  • z is in scope after its declaration until end of if 

void f (int x) {
  int y = x + 1;
  if (x > y) {
    int z = y + 1;
    printf ("z = %d\n", z);
  }
}

Terminology: Occurrences

  • Occurrences of identifiers classified as one of
    • free occurrence has no matching binding 
      
y = 5*x;   // Free occurrences of x and y
    • binding occurrence declares the identifier 
      
int y;    // binding occurrence of y
    • bound occurrence follows matching declaration 
      
int y;    // Binding occurrence of y
int x;    // Binding occurrence of x

x=6;      // Bound occurrence of x
y = 5*x;  // Bound occurrences of x and y

Terminology: Occurrences

  • Complete programs usually have no free occurrences of identifiers
  • How do IDEs treat free occurrences?

Not Just Variables

  • Applies to identifiers for
    • normal variables
    • function parameters
    • function type parameters
    • function/method names
    • class names
    • and more

Forward Declarations

  • C, C++ require forward declarations
  • Most other modern languages do not 

char f (int x);

char g (int x) {
  return f (x) + f (x);
}

char f (int x) {
  if (x > 0) { 
    return g (x >> 8);
  } else {
    return 1;
  }
}

Shadowing - Java


public class C {
  static void f () {
    int x = 1;
    {
      int y = x + 1;
      {
        int x = y + 1;
        System.out.println ("x = " + x);
      }
    }
  }
}
          

$ javac C.java 
C.java:7: error: variable x is already defined in method f()
        int x = y + 1;
            ^
1 error

Shadowing - Java

  • Fields have different treatment 

public class C {
  static int x = 1;
  static void f () {
    int y = x + 1;
    {
      int x = y + 1;
      System.out.println ("x = " + x);
    }
  }
  public static void main (String[] args) {
    f ();
  }
}
          

$ javac C.java 
$ java C
x = 3

Shadowing C

  • C is less strict than Java (on shadowing) 

int main () {
  int x = 1;
  {
    int y = x + 1;
    {
      int x = y + 1;
      printf ("x = %d\n", x);
    }
  }
}
          

$ gcc -o scope scope.c 
$ ./scope
x = 3

Shadowing Scala

  • Scala is less strict than Java (on shadowing) 

object C:
  def f () =
    var x = 1;
      var y = x + 1;
        var x = y + 1;
        println ("x = " + x)
  def main (args:Array[String]) = 
    f ()
          

$ scalac C.scala 
$ scala C
x = 3

Shadowing Scala

  • Shadowing occurs in the REPL naturally 

scala> val x = 1
x: Int = 1

scala> def f (a:Int) = x + a
f: (a: Int)Int

scala> f (10)
res0: Int = 11

scala> val x = 2
x: Int = 2

scala> x
res1: Int = 2

scala> f (10)
res2: Int = 11

Shadowing Scala

  • REPL behavior corresponds to 

{
  val x = 1;
  def f (a:Int) = x + a
  f (10)
  {
    val x = 2; 
    x
    f (10)
    ...
  }
}

Shadowing and Recursion

  • Is x in scope? 

int main (void) {
  int x = 10;
  {
    int x = x + 1;
    printf ("x = %08x\n", x);
  }
  return 0;
}
          

$ gcc -o scope scope.c

$ gcc -Wall -o scope scope.c
scope.c: In function ‘main’:
scope.c:5:7: warning: unused variable ‘x’ [-Wunused-variable]
scope.c:7:9: warning: ‘x’ is used uninitialized in this function [-Wuninitialized]

$ ./scope 
x = 00000001

Lifetime

  • Lifetime of an area of memory
    • duration during which it is allocated
  • READING: chapter 7 of Mitchell textbook
    • recall activation records from Systems I

Activation Records

  • Activation records: storage space for local variables / intermediate values that the runtime system generates
    • also known as stack frames
  • ARs almost always placed on a call stack

Typical Storage Options

  • global (static)
    • available for lifetime of program
  • in AR in call stack (stack-allocated)
    • available whilst function active
      (called but not returned)
  • in heap (heap-allocated)
    • available until deallocated
      (manually or via garbage collection)

Storage Lifetime Issues

  • Lifetime too short
    • reads return other value
    • writes overwrite other value
    • resource state incorrect, e.g., file handle closed
    • can cause security problems
  • Lifetime too long
    • uses too much memory (memory leak)
    • too late in freeing other resources / finalization

Control Links

  • Some systems, e.g., 32-bit x86, use control links
  • control link in each AR points to previous AR
  • Control links provide linked list / stack view of ARs
  • ebp register points to AR for most recently called function

Call Stack of ARs

  • Call stack of ARs allows
    • fast allocation of fresh AR on function call
    • fast deallocation of AR on function return
    • contrast with heap allocation
  • Stack discipline
    1. (call f) allocate AR for f
    2. (call g) allocate AR for g
    3. (return from g) deallocate AR for g
    4. (return from f) deallocate AR for f

Multiple Threads

  • Each thread needs a separate call stack
  • Calls and returns in separate threads are independent

Heap Allocation

  • Heap allocation can use other patterns
    1. allocate M bytes
    2. allocate N bytes
    3. deallocate M bytes
    4. deallocate N bytes
  • Allocations may be long-lived, others short-lived
  • Gives freedom, but more costly than call stack

Common Problems

  • PLs with garbage collection
    • lifetime too long (not GCed)
  • For Java, Scala, C#, Python, Ruby, JS, Scheme, etc.

Common Problems

  • PLs with manual memory management (C, C++)
    • pointers to storage whose lifetime has ended
      • dangling pointers to an old AR
      • dangling pointers to freed heap memory
        (use after free)
      • double freeing of heap memory

Dangling Pointer - AR


#include <stdio.h>
#include <stdlib.h>

int *f (int x) {
  int y = x;
  return &y;
}

int main (void) {
  int *p = f (1);
  printf ("*p = %d\n", *p);
  return 0;
}
          

$ gcc -o ar ar.c
ar.c: In function ‘f’:
ar.c:6:3: warning: function returns address of local variable 
  [enabled by default]

$ ./ar 
*p = 1

Dangling Pointer - AR


$ valgrind ./ar
==5505== Memcheck, a memory error detector
==5505== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==5505== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==5505== Command: ./ar
==5505== 
==5505== Conditional jump or move depends on uninitialised value(s)
==5505==    at 0x4E7C1A1: vfprintf (vfprintf.c:1596)
==5505==    by 0x4E85298: printf (printf.c:35)
==5505==    by 0x400536: main (in /tmp/ar)
==5505== 
==5505== Use of uninitialised value of size 8
==5505==    at 0x4E7A49B: _itoa_word (_itoa.c:195)
==5505==    by 0x4E7C4E7: vfprintf (vfprintf.c:1596)
==5505==    by 0x4E85298: printf (printf.c:35)
==5505==    by 0x400536: main (in /tmp/ar)
==5505== 
==5505== Conditional jump or move depends on uninitialised value(s)
==5505==    at 0x4E7A4A5: _itoa_word (_itoa.c:195)
==5505==    by 0x4E7C4E7: vfprintf (vfprintf.c:1596)
==5505==    by 0x4E85298: printf (printf.c:35)
==5505==    by 0x400536: main (in /tmp/ar)
==5505== 
*p = 1
==5505== 
==5505== HEAP SUMMARY:
==5505==     in use at exit: 0 bytes in 0 blocks
==5505==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==5505== 
==5505== All heap blocks were freed -- no leaks are possible
==5505== 
==5505== For counts of detected and suppressed errors, rerun with: -v
==5505== Use --track-origins=yes to see where uninitialised values come from
==5505== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)

Not Dangling Pointer - AR


#include <stdio.h>
#include <stdlib.h>

int *f (int x) {
  int *result = (int *) malloc (sizeof (int));
  *result = x;
  return result;
}

int main (void) {
  int *p = f (1);
  printf ("*p = %d\n", *p);
  return 0;
}
          

$ gcc -Wall -o ar ar.c

$ ./ar 
*p = 1

Not Dangling Pointer - AR


$ valgrind ./ar
==10962== Memcheck, a memory error detector
==10962== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==10962== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==10962== Command: ./ar
==10962== 
*p = 1
==10962== 
==10962== HEAP SUMMARY:
==10962==     in use at exit: 4 bytes in 1 blocks
==10962==   total heap usage: 1 allocs, 0 frees, 4 bytes allocated
==10962== 
==10962== LEAK SUMMARY:
==10962==    definitely lost: 4 bytes in 1 blocks
==10962==    indirectly lost: 0 bytes in 0 blocks
==10962==      possibly lost: 0 bytes in 0 blocks
==10962==    still reachable: 0 bytes in 0 blocks
==10962==         suppressed: 0 bytes in 0 blocks
==10962== Rerun with --leak-check=full to see details of leaked memory
==10962== 
==10962== For counts of detected and suppressed errors, rerun with: -v
==10962== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)

Dangling Pointer - Heap


#include <stdio.h>
#include <stdlib.h>

int *f (int x) {
  int *result = (int *) malloc (sizeof (int));
  *result = x;
  return result;
}

int main (void) {
  int *p = f (1);
  free (p);
  printf ("*p = %d\n", *p);
  return 0;
}
          

$ gcc -Wall -o ar ar.c

$ ./ar 
*p = 0

Dangling Pointer - Heap


$ valgrind ./ar
==13594== Memcheck, a memory error detector
==13594== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==13594== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==13594== Command: ./ar
==13594== 
==13594== Invalid read of size 4
==13594==    at 0x4005D2: main (in /tmp/ar)
==13594==  Address 0x51f0040 is 0 bytes inside a block of size 4 free'd
==13594==    at 0x4C2A82E: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13594==    by 0x4005CD: main (in /tmp/ar)
==13594== 
*p = 1
==13594== 
==13594== HEAP SUMMARY:
==13594==     in use at exit: 0 bytes in 0 blocks
==13594==   total heap usage: 1 allocs, 1 frees, 4 bytes allocated
==13594== 
==13594== All heap blocks were freed -- no leaks are possible
==13594== 
==13594== For counts of detected and suppressed errors, rerun with: -v
==13594== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)

More Scope

  • Recursive definitions
  • Static and dynamic scope

C Initialization

  • Which x in the initializer, x + 1 

int main (void) {
  int x = 10;
  {
    int x = x + 1;
    printf ("x = %08x\n", x);
  }
  return 0;
}
          

$ gcc -o scope scope.c

$ gcc -Wall -o scope scope.c
scope.c: In function ‘main’:
scope.c:5:7: warning: unused variable ‘x’ [-Wunused-variable]
scope.c:7:9: warning: ‘x’ is used uninitialized in this function [-Wuninitialized]

$ ./scope 
x = 00000001

Java Initialization

  • Java requires that all variables be initialized before use. 

class C {    
    public static void main (String[] args) {
        int x = 1 + x;
        System.out.printf ("x = %08x\n", x);
    }    
}

x.java:3: error: variable x might not have been initialized
        int x = 1 + x;
                    ^
1 error

Scala Initialization

  • Scala variables and fields are set to 0 before the initialization code is run
  • Recursion is allowed when initializing fields 

scala> val x:Int = 1 + x 
x: Int = 1

scala> :javap -p -c -filter -
Compiled from "<console>"
public class  {
private final int x;
public int x();
  0: aload_0
  1: getfield      #22                 // Field x:I
  4: ireturn

public ();
  ...
  10: invokevirtual #28                 // Method x:()I
  13: iconst_1
  14: iadd
  15: putfield      #22                 // Field x:I
  ...
}

Scala Initialization

  • Try with a list
  • Null pointer exception happening on print, since null != Nil 

scala> val xs:List[Int] = 1 :: xs
java.lang.NullPointerException
  ... 28 elided

Scala Initialization

  • Try to code these as our own objects 

scala> case class S(head:Int, tail:S)
defined class S

scala> val ss:S = S(1, ss)
ss: S = S(1,null)
  • Need to delay evaluation of tail 

scala> case class T(head:Int, tail:()=>T)
defined class T

scala> val ts:T = T(1, ()=>ts)
ts: T = T(1,$$Lambda$1324/2038353966@4d500865)

scala> ts.tail().tail().head
res14: Int = 1

Scala Streams

  • Streams do this for us!
  • #:: is non-strict in right-hand argument 

scala> val ones:Stream[Int] = 1 #:: ones
ones: Stream[Int] = Stream(1, ?)

scala> ones.take (5)
res0: scala.collection.immutable.Stream[Int] = Stream(1, ?)

scala> ones.take (5).toList
res1: List[Int] = List(1, 1, 1, 1, 1)

Scala Streams

  • Lazy evaluation of stream elements 

scala> def f (x:Int) : Stream[Int] = { { println (s"f(${x})"); x } #:: f(x+1) }
f: (x: Int)Stream[Int]

scala> val xs:Stream[Int] = f(10)
f(10)
xs: Stream[Int] = Stream(10, <not computed>)

scala> xs.take(4).toList
f(11)
f(12)
f(13)
res12: List[Int] = List(10, 11, 12, 13)

scala> xs.take(4).toList
res13: List[Int] = List(10, 11, 12, 13)

scala> xs.take(6).toList
f(14)
f(15)
res14: List[Int] = List(10, 11, 12, 13, 14, 15)

Scala Lazy Lists

  • Lazy evaluation of list elements 

scala> def f (x:Int) : LazyList[Int] = { { println (s"f(${x})"); x } #:: f(x+1) }
f: (x: Int)LazyList[Int]

scala> val xs:LazyList[Int] = f(10)
xs: LazyList[Int] = <not computed>

scala> xs.take(4).toList
f(10)
f(11)
f(12)
f(13)
res12: List[Int] = List(10, 11, 12, 13)

scala> xs.take(4).toList
res13: List[Int] = List(10, 11, 12, 13)

scala> xs.take(6).toList
f(14)
f(15)
res14: List[Int] = List(10, 11, 12, 13, 14, 15)

Dynamic and Static Scope

What Does This Program Do?

  • What does this program do?
  • Using Scala syntax, but various different semantics
    • not just Scala's semantics 

var x:Int = 10
def foo () = 
  x = 20

def bar () = 
  var x:Int = 30
  foo ()

bar ()
println (x)

Static Scope

  • Static scope: identifiers are bound to the closest binding occurrence in an enclosing block of the program code
  • Static scoping property: We can rename any identifier, so long as we rename it consistently throughout its scope
    (and so long as the new name we have chosen does not appear in the scope)
  • Also known as lexical scope

Dynamic Scope

  • Dynamic scope: identifiers are bound to the binding occurrence in the closest activation record.
  • Consistent renaming may break a working program!

Dynamic Scope

  • Where could z come from? 
    
...
def g (x:Int) : Int = 
  var y:Int = x * 2
  z * x * y          // x and y are local; z is non-local
  • Dynamic scope:
    • non-locals are not resolved (bound) until runtime
    • to resolve non-local identifier, look at the callers

Static vs Dynamic Scope

  • Scala uses static scope (prints 20)
  • Most languages do use static scope 

var x:Int = 10
def foo () : Unit =
  x = 20

def bar () : Unit = 
  var x:Int = 30
  foo ()

bar ()
println (x)

Static vs Dynamic Scope

Bash (prints 10): 


x=10
function foo() {
    x=20
}
function bar() {
    local x=30
    foo
}
bar
echo $x

Static vs Dynamic Scope

C functions (prints 20): 


int x = 10;
void foo () {
    x = 20;
}
void bar () {
    int x = 30;
    foo ();
}
int main () {
    bar ();
    printf ("x=%d\n", x);
}

Static vs Dynamic Scope

C macros (prints 10): 


int x = 10;
#define foo() { \
  x = 20; \
}
#define bar() { \
    int x = 30; \
    foo (); \
}
int main () {
    bar ();
    printf ("x=%d\n", x);
}

Static vs Dynamic Scope

Python (prints 20): 


def main():
  def foo ():
    nonlocal x
    x = 20

  def bar ():
    x = 30
    foo ()

  x = 10
  bar ()
  print (x)

main()

Static vs Dynamic Scope

Python (prints 20): 


def foo ():
  global x
  x = 20

def bar ():
  x = 30
  foo ()

x = 10
def main():
  bar ()
  print (x)

main()

Python

Python global scope is not static 


def useX():
  print (x)

def defX():
  global x
  x = 1
        

>>> useX()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in useX
NameError: name 'x' is not defined
>>> defX()
>>> useX()
1

Static vs Dynamic Scope

  • Well-known PLs have included dynamic scoping...
    • Lisp, Perl, ...
  • ...and later added static scoping!

Static vs Dynamic Scope

Emacs Lisp (prints "10"): 


(let ((x 10))
  (defun foo ()
    (setq x 20))
  (defun bar ()
    (let ((x 30))
      (foo)))
  (bar)
  (message (int-to-string x)))

Static vs Dynamic Scope

Common Lisp (prints 20): 


(let ((x 10))
  (defun foo ()
    (setq x 20))
  (defun bar ()
    (let ((x 30))
      (foo)))
  (bar)
  (print x))

Static vs Dynamic Scope

Scheme (prints 20): 


(let ((x 10))
  (define (foo)
    (set! x 20))
  (define (bar)
    (let ((x 30))
      (foo)))
  (bar)
  (display x)
  (newline))

Static vs Dynamic Scope

Perl (prints 10): 


local $x = 10;
sub foo {
    $x = 20;
}
sub bar {
    local $x = 30;
    foo ();
}
bar ();
print ($x);

Static vs Dynamic Scope

Perl (prints 20): 


my $x = 10;
sub foo {
    $x = 20;
}
sub bar {
    my $x = 30;
    foo ();
}
bar ();
print ($x);