The Design and Analysis of a True Random Number Generator in a Field Programmable Gate Array

Transcription

1 The Design and Analysis of a True Random Number Generator in a Field Programmable Gate Array A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science at George Mason University By Paul W. Kohlbrenner Bachelor of Science Carnegie Mellon University, 1984 Director: Kris Gaj, Professor Department of Electrical and Computer Engineering Fall Semester 2003 George Mason University Fairfax, Virginia

2 Copyright 2003 Paul W. Kohlbrenner All Rights Reserved ii

3 iii DEDICATION This is dedicated to my wife Marianne, without whose support this project would never have been possible.

4 iv ACKNOWLEDGEMENTS I would like to thank Kris Gaj for encouraging me to pursue this project and for his sense of humor about the difficulties I have had in finishing it. I would also like to thank the good taxpayers of Virginia for their wisdom in supporting George Mason University.

5 v TABLE OF CONTENTS Page Abstract...viii 1. INTRODUCTION The Need for Random Bits in Cryptographic Processes Field Programmable Gate Arrays Thesis goals BACKGROUND What is a RNG? Kinds of RNGs FPGAS AND CRYPTOGRAPHIC PROCESSES TESTING TRNGS PRNG vs. TRNG Testing The Chi-Squared Test Statistical Tests Certification TRNG BACKGROUND Sources of randomness Previous Work The Intel RNG The VIA C3 Nehemiah RNG MY TRNG DESIGN Overview The Ring Oscillator The Sampler The Control Circuits Ring Oscillator Issues Evidence of Jitter Bias in the Output Bit Generation Speeds Testing CONCLUSION Future Work Goals met Appendix List Of References... 69

6 vi LIST OF TABLES Page Table 1 - Chi-Squared Example Sample Values Table 2 - XOR Bias Improvement Table 3 - Results from NIST Test Suite... 56

7 vii LIST OF FIGURES Page Figure 1 - RNG Properties... 8 Figure 2 - Period Jitter Figure 3 - Chi-Squared example categories Figure 4 - ATT Patent Figure 5 - TRNG Design...38 Figure 6 - The ring oscillator circuit Figure 7 - CLB Layout Figure 8 - The sampler circuit Figure 9 - Timing diagram for the sampler circuit Figure 10 - Sampler circuit's behavior with jitter Figure 11 - Normalized CLB speeds for rows Figure 12 - Signal S0 cycle lengths for a ring oscillator difference of 35 ps Figure 13 - Jitter-less signal timing Figure 14 - Signal S0 cycle lengths for a ring oscillator difference of 22 ps... 50

8 1. INTRODUCTION 1.1 The Need for Random Bits in Cryptographic Processes Cryptographic systems need strings of randomly chosen bits in order to function correctly. The most obvious need is with key generation. A secret key must be un-guessable for it to be secure. If a Random Number Generator (RNG) meets this fundamental requirement an attacker will be forced to try every possible combination of bits for the given key length. Other consumers of random bits in a cryptographic system include: Initialization vectors Block padding Challenges Many cryptographic systems have a single subsystem that supplies these crucial bits. In some cases the random bits are revealed to observers (e.g. the initialization vector is transmitted in the clear in many protocols). Thus an attacker can observe the 1

9 2 characteristics of the bits produced by the system s RNG and take advantage of any weakness found there. RNGs used for cryptographic processes must, therefore, be considered a critical part of the cryptographic system [13]. A weakness or failure in the RNG can lead to a complete failure of the system [17]. RNGs are divided into two basic types. RNGs that base their output on a physical source of randomness are known as True Random Number Generators (TRNGs). RNGs that are given an initial random seed and thereafter generate random-seeming numbers in a deterministic way are known as Pseudo Random Number Generators (PRNGs). 1.2 Field Programmable Gate Arrays A Field Programmable Gate Array (FPGA) is an electronic component that allows a designer to reconfigure the device to implement a wide variety of functions. FPGAs consist of an array of Configurable Logic Blocks (CLBs) that can be interconnected via a programmable switching array. Each CLB includes two flip-flops and two four-input lookup tables that can be used to implement two arbitrary functions of four variables. A design can be implemented by a netlist of lookup tables and flip-flops. Newer models of FPGAs contain elements in addition to the CLB, including blocks of RAM, Phase Locked Loops (PLLs), Delay Locked Loops (DLLs), and even whole processor cores. The connections between elements are described using a high-level language. A suite of

10 3 tools is used to convert the description into a bitstream that can be sent to the FPGA to program it. 1.3 Thesis goals The primary goal of this thesis project was to: Design and build a TRNG in an FPGA with the following characteristics: o The TRNG uses only the standard CLBs in the FPGA. o The output bits pass the standard statistical tests of randomness. o The output bit rate is acceptable for cryptographic applications. Secondary goals of the project included: Assess the suitability of the current generation of FPGA design tools for creating and testing RNGs on FPGAs. Investigate the state of the publicly available randomness testing suites. This thesis will show that I was successful in accomplishing all of these goals.

11 4 1.4 Thesis organization Chapter 2 of this thesis describes what a RNG is and how different RNGs are categorized. Chapter 3 describes FPGAs and explains why they are good platforms for cryptographic systems. In Chapter 4, I show how one can test the different kinds of RNGs. In Chapter 5, I cover the various sources of randomness and I describe several commercially available RNGs. Chapter 6 details the TRNG that I built and tested. Finally, in chapter 7, I discuss some possible future work on my TRNG.

12 2. BACKGROUND 2.1 What is a RNG? Humans have used RNGs for thousands of years [7]. Any game that involves chance generally requires a RNG. Consider the common six-sided die; most people have an intuitive understanding that each side of the die should appear about 1/6th of the time. Over time the device should produce a uniform distribution of the numbers between 1 and 6. A secondary understanding is that for any given six rolls it would not be surprising if one number didn't come up. People understand that small data samples aren't good at predicting the overall fairness of the device. In fact, it is precisely this small-scale uneven distribution that makes the device useful for a game of chance. On the other hand a die that rolls a six 20 times in a row would have its fairness questioned. From an intuitive standpoint then: A RNG is a device that produces a stream of numbers each of which is a surprise, but over the long run the numbers should follow a specified distribution. Bruce Schneier, in his book Applied Cryptography [37], uses the following three part definition of a RNG: 5

13 6 1. The output looks random. This means that it passes all the statistical tests of randomness that we can find. 2. It is unpredictable. It must be computationally infeasible to predict what the next random bit will be, given complete knowledge of the algorithm, the hardware generating the sequence, and all of the previously generated bits. 3. It cannot be reliably reproduced. If you run the sequence generator twice with the exact same input, you will get two completely unrelated random sequences. Depending on our intended use we might want a RNG that has only some of these properties. For example, a system that attempts to simulate a physical process might want a generator that produces a statistically acceptable sequence. The added overhead of making the sequence unpredictable is of no use to the system. And worse, if the sequence can't be reproduced then simulations cannot be rerun with the same input data. Cryptographic systems always require RNGs with the first two properties and sometimes with the third property. 2.2 Kinds of RNGs RNGs used in data processing can been divided into three general categories:

14 7 Pseudo Random Number Generators (PRNGs): These generators are algorithms that are initialized with an externally generated sequence and produce a much longer sequence that appears to be random. After being initialized with a seed value, the internal state of the generator completely determines the next bit to be generated. Given the same seed value a PRNG will always produce the same sequence. Cryptographically Secure Pseudo Random Number Generators (CSPRNGs): If, given all the previous output from a PRNG, and the complete algorithm being used in the PRNG, it is computationally infeasible to predict the next output, then the PRNG is considered cryptographically secure. In this context, computationally infeasible means that you (or your adversary) do not have the computational resources to successfully attack the problem using a brute-force method. CSPRNGs are used in stream ciphers to generate the key-stream. True Random Number Generators (TRNGs): These generators base their output entirely on an underlying random physical process. Unlike their deterministic cousins the output is based only on the physical process and not any previously produced bits. Often the raw bits generated by the physical source are biased (the probability of a '1' is not 0.5), and thus some bias reduction is necessary. Not coincidentally, the three types of RNGs can be seen to have different sets of Schneier s properties (Figure 1).

15 8 TRNG Cannot be Reproduced CSPRNG Unpredictable Unpredictable PRNG Looks Random Looks Random Looks Random Figure 1 - RNG Properties PRNGs have only the first of Schneier s properties. CSPRNGs have both the first and the second. TRNGs have all three properties. Another useful way to analyze a RNG is to examine how its internal state is kept. If the RNG generates the next output from the previously revealed output it can only be a PRNG (an adversary just has to run the algorithm on the previous output). If the output is based on some hidden internal state that is not usefully leaked in the previous output then the RNG may be a CSPRNG. In many cases it is possible to combine the two kinds of RNGs and produce a useful hybrid. In cases where the output rate of a physical source of random bits is lower then

16 9 the desired output bit rate of the RNG it is possible to periodically re-seed a PRNG with bits from the TRNG to achieve an acceptable output [22].

17 3. FPGAS AND CRYPTOGRAPHIC PROCESSES FPGAs are emerging as an attractive platform for cryptographic implementations. Now fast enough and large enough to implement any cryptographic algorithm, they offer benefits such as: Near-ASIC encryption speeds With an FPGA one can achieve close to the same encryption speeds as an ASIC design. Additionally, a direct hardware implementation of an algorithm is often much faster than a software implementation running on a general purpose CPU. Algorithm and resource efficiencies Many cryptographic systems offer several parameterizations for different levels of security (key length is the best example). A designer who knows a given parameterization ahead of time can produce a more efficient design (in space and/or energy requirements). By storing several designs with various expected parameterizations the lowest cost design can be used. Many algorithms require that data be processed sequentially through several different sub-processes. If pipelining is not possible (or needed) than the non- 10

18 11 active hardware associated with each sub-process is idle. An FPGA can be reconfigured for each sub-process resulting in more efficient use of the available hardware. In service algorithm modification Until programmable logic devices became available a data processing algorithm that was realized in hardware was fixed in its operation. The resulting increase in speed of a hardware solution was balanced against this inability to fix or upgrade the algorithm. This modification benefit is crucial in cryptographic systems because a problem discovered in the algorithm can lead to the complete loss of security (and therefore of function) of the system. In FPGA designs the modified implementation can be downloaded to the device (perhaps remotely). Low development costs When compared to the ASIC design cycle, FPGAs are a much lower cost alternative for development of many systems. The design tools used for FPGAs are often an order of magnitude cheaper than ASIC design tools. FPGA designs can be tested (and fielded) immediately where as ASICs must be sent to a foundry. However, in some high-volume situations it may be that ASICs result in a lower total cost.

19 12 Parameter and algorithm eraser on intrusion detection In some cryptographic systems the underlying algorithms are secret or proprietary. In these systems it may be required that the device resist attempts to extract the algorithm. An FPGA can be completely erased (usually by removing the power to the device). A physical intrusion detection system (IDS) could be designed that would, non-destructively, erase the algorithm upon detection of an attempted compromise. The device could be restored to full function later by simply downloading the FPGA configuration. An IDS that does not cause a device to permanently self-destruct can be made much more sensitive. Until recently, FPGA designs that included a cryptographic component and required a source of random bits had limited options. The designer could use any of a variety of special purpose TRNG chips and make the necessary physical connections. However, these external interconnections are weak points that an attacker could observe and exploit. Or the designer could implement a PRNG in the FPGA and suffer the resulting degradation in security [8]. In [16] a third option was introduced. By carefully engineering the frequency of two clock signals the non-deterministic jitter present in all digital signals could be extracted. Jitter is defined by the ITU-T as the variations in the significant instants of a clock or data signal [19]. Jitter in digital circuits has many sources including semiconductor noise, cross talk, power supply variations, and electro-magnetic fields in the operating environment. Semiconductor noise is the non-deterministic component that I based my

20 13 construction on. There are several ways to characterize jitter. Period jitter, which is the measure of deviation in a clock's period from its average period, is shown in Figure 2. Average Period Jitter Uncertainty about the exact timing of the rising edge. Figure 2 - Period Jitter The extraction technique presented in [16] uses one clock signal to sample the value of a second clock signal on each cycle. If the two clock frequencies are different, the point sampled in the second signal will change with each sampling cycle. If the change is small enough it will eventually sample the second signal in the jitter zone. Thus the sampling will produce a large number of deterministic bits and at least one uncertain bit taken in the jitter zone. XORing the deterministic bits and the non-deterministic bit(s) produces a single random bit. A Phase Locked Loop (PLL) present on Altera FPGAs was used to produce the two clock signals used in this technique. The PLL synthesized the new signals from the system clock. A PLL is a device that contains an oscillator whose frequency is adjusted such

21 14 that there is no phase difference between it and the input clock signal. PLLs in FPGAs have two primary uses: Reduce clock skew in large clock distribution nets. Frequency synthesis. Frequency synthesis is accomplished by modifying the oscillator signal before it is fed to the phase detector thereby causing the internal oscillator to increase or decrease the frequency of its output signal. Very fine control of the output frequency of the PLL is possible. Xilinx is the largest manufacturer of FPGAs. With a 44% share of the Programmable Logic Device (PLD) market [40] and a broad line of FPGAs and other programmable logic devices, Xilinx is often the choice of system architects. Unfortunately (for my application), most Xilinx FPGAs provide Delay Locked Loops (DLLs) instead of PLLs. A DLL inserts delay elements into the path of the clock signal until the phase difference of the incoming clock and a one cycle delayed clock is zero. While DLLs work well to reduce clock skew (their primary function), they cannot provide the frequency synthesis necessary for my application.

22 4. TESTING TRNGS 4.1 PRNG vs. TRNG Testing In section 2.2 I discussed the two major categories of RNGs: Pseudo Random Number Generators (PRNGs and CSPRNGs) True Random Number Generators (TRNGs) The output of both kinds of devices is tested using a variety of statistical tests (described in the next section). TRNGs must be further analyzed to make sure that the required physical source of randomness is functioning as expected. This additional testing for TRNGs usually involves examining internal signals of the TRNG and describing ways of detecting the failure of the source of randomness during normal operation [35]. 4.2 The Chi-Squared Test Many tests for randomness use the Chi-Squared test to analyze a given set of results. The Chi-Squared test can indicate how far a given sample is from a perfectly random result. The mathematical derivation of the Chi-Squared test is complex and the reader is referred to the excellent explanation given in [24]. However the test itself is relatively easy to 15

23 16 explain. Divide the sequence to be tested into independent subsequences. Categorize each subsequence and count the number of subsequences in each category. Calculate the following statistic: V = k s= 1 ( Y np ) s np s s 2 where: k The number of categories n - The number of subsequences Y s - The number of subsequences found in each category ps - The probability that a subsequence would be in category s Thus V is just the sum of the squares of the difference between the number of subsequences found in each category and the expected number of subsequences in a category (divided by the number of subsequences in a category). We can consider 2 ( Ys np s ) the square of the distance of each count from the ideal count. Clearly, if the sequence had exactly the number of expected subsequences in each category than the individual distances would be zero and the statistic V would equal zero. As shown in [24] for large enough values of n, the V statistic is distributed according to the Chi- Square distribution. By consulting a table of Chi-Square values for a given number of categories we can determine how closely our sequence matches an ideal sequence. The number of categories (minus one) is called the degrees of freedom of the test.

24 17 For example, as shown in Figure 3 divide a sequence of binary values in to groups of three bits Figure 3 - Chi-Squared example categories. Each group of three bits will form a number between zero and seven (thus, k=8). Count the number of groups that form each of the eight numbers. For this example, assume we have 30,000 bits (n = 30,000/3 = 10,000) and that the counts occur as follows: Table 1 - Chi-Squared Example Sample Values Category Count

25 18 A perfectly random sequence would have n/8 sequences in each category (thus p = 1 8 ). Given the set of values shown in Table 1 we find the V statistic to be Consulting a Chi-Squared table for 8-1 = 7 degrees of freedom we find that less than 1% of sequences would have a V statistic this high. If we test a number of 30,000 bit sequences and find about 1% of sequences similar to our example we would conclude that the generator was good. If, on the other hand, we consistently got V statistics that were this high we would conclude that the generator was probably flawed. s The above example illustrates several points: 1. Given a large enough n, the Chi-Square test can be very sensitive. The counts shown in Table 1 appear to be close to the expected 1,250 and yet we see that very few random sequences would have a V statistic so high. 2. Some sequences will appear to be non-random. We expect that 1% of the sequences will have a V statistic above the 1% level (by definition). Thus one failed test for one sequence does not indicate a bad generator. 3. The Chi-Squared test will not detect short sequences of non-random behavior. If in the 30,000 bits we were testing the first 15,000 had no sequences of 0,0,1 but the next 15,000 bits had twice as many as would be expected the test would not detect it. Thus the Chi-Squared test must be run with varying sized sets of data.

26 Statistical Tests Some of the public domain suites for testing RNGs include: The DIEHARD tests These tests were written George Marsaglia, a professor at Florida State University (now retired). The suite consists of 15 tests that each examines a different expected property of a sequence of randomly generated bits [27] [28]. For example, the description for the Binary Rank Test is as follows: This is the BINARY RANK TEST for 32x32 matrices. A random 32x32 binary matrix is formed, each row a 32-bit random integer. The rank is determined. That rank can be from 0 to 32, ranks less than 29 are rare, and their counts are pooled with those for rank 29. Ranks are found for 40,000 such random matrices and a chisquare test is performed on counts for ranks 32, 31, 30 and <=29. The suite requires at least 80 million bits (10 Megabytes) to run correctly. Originally written in FORTRAN and later mechanically converted to C, the suite is difficult to compile locally. Fortunately the download site has a precompiled version for DOS.

27 20 The NIST test suite is officially known as A Statistical Test Suite For Random and Pseudorandom Number Generators for Cryptographic Applications (Pub with revisions). The final version of this suite was released in May The suite consists of 16 tests each of which tests a different aspect of the input random sequence. The suite comes with a well-written user s manual that gives detailed descriptions of each test together with the mathematical theory behind each test [33]. The source code for the suite is written in C and originally targeted for a Sun system (running Solaris). Although the NIST web site claims it is possible to compile the system on a Windows machine I never completed the modifications necessary to successfully run the entire suite under Windows. The suite does compile and appear to run correctly on the GMU CPE02 Solaris system. Running the complete NIST suite requires the input of a number of parameters for each of the tests. Results reported from running the suite should always list the parameters used. Although these two suites are quite different in concept and construction, in my experience sequences that fail one suite will invariably fail the other. The DIEHARD suite generally runs quickly and tests only the first 80 million or so bits in the input file. The NIST suite can be parameterized to test very large bit samples but it tends to run very

28 21 slowly. The results presented later in this report of running the NIST suite on 128Mbytes of data took approximately three days of CPU time on the CPE02 system. There are several suites that are commercially available for testing random outputs. ComScire, company that manufactures TRNGs, has recently released a test suite called the RNGmeter. The program has a GUI interface and will analyze files of bits produced by a TRNG/PRNG and it can do real-time testing on a bitstream from a TRNG. The suite uses five basic tests that it runs continuously. The testing stops when any of the five tests fail. The GUI records the number of bits that pass before a failure (or when it reaches the end of the input file) in the form of a logarithmic score. The tests used are: 1. I/O Balance the number of ones vs. zeroes in the block being tested. 2. Auto Correlation shifts of the bitstream by up to 32 bits. 3. Entropy Test a modified version of Maurer s Universal Test. 4. Serial Test From a paper by I J Good entitled: The serial test for sampling numbers and other tests for randomness, in the Proc. Cambridge Philosophical Soc. 49, , OQSO - Overlapping-Quadruples-Sparse-Occupancy test (from the Diehard suite).

29 22 The RNGmeter program (EXE only) is available at no cost from ComScire (see A second suite is available from the Queensland University of Technology in Australia. The suite is known as crypt-x and was designed to test the output of encryption algorithms. Since a properly encrypted message should look random, crypt-x also works well for testing RNGs. They do not seem to offer free versions of the suite (a single academic user license is 600 AUD). More information is available at: While I found the above test suites are useful for doing formal testing of large data sets. I found I also needed a set of tools to be able to quickly assess the merit of smaller samples. To that end I wrote a set of four tests and some support code that allowed me to flexibly test small to medium (20,000 10,000,000 bits) samples. My test system allows one to break up the input file into segments and report separately on each segment (the default is to have just one segment consisting of the entire file). The four tests I chose to implement were: 1. Monobit test This test counts the number of ones and zeros in the given segment. I performed a Chi-Squared test with one degree of freedom on the resulting counts. The test reports the test statistic and a pass or fail message (fail below the 5% or above the 95% confidence level).

30 23 2. Poker test This poker test 1 divides the input sequence into k nonoverlapping groups of two or more bits. The number of each unique m permutation is counted (there are 2 kinds, where m is the number of bits and n i is the number of instances of each pattern found) and a Chi-Squared m test with 2 1 degrees of freedom is calculated using the statistic: 2 k m m 2 i= 1 n 2 i k I report pass if the statistic is above the 5% and below the 95% confidence interval. Other suites (including the NIST suite) choose a fixed m (in the past FIPS specified m =4). In my suite I run this test for all m such that 2 m M where M allows for at least eight expected counts of each unique permutation. 3. Runs and Gaps test This test counts the number and size of the runs (contiguous ones) and gaps (contiguous zeros). The test prints out the counts for inspection. While I don t calculate it, the statistic given in the HOAC is: 1 Interestingly, there are two different poker tests described in the literature. I implemented the one described in The Handbook of Applied Cryptography, 5.4.4(iii) [29]. The other well-known one is described by Knuth in The Art of Computer Programming, Volume 2 Seminumerical Algorithms (Third Edition), D [24]. In Knuth s test one counts the number of distinct values in successive nonoverlapping groups of five integers drawn from the test sequence and applies a Chi-Squared test to a calculated statistic.

31 24 k i= 1 2 k ( B e ) ( G e ) i e i i + i= 1 i e i i 2 Where: Bi and G i - the actual number of blocks and gaps of length i. i+ 2 e - the expected number of blocks/gaps of length i ( e = ( n i + 3) / 2 ) i i n - the number of bits to be tested k - the largest integer i for which e i 5 This statistic is approximately Chi-Squared with 2k 2 degrees of freedom. 4. Autocorrelation test This test counts the misses (different bit values in the same position) between the test sequence and a shifted version of the same sequence. My test tries all shifts of 1 d 16 bits. I calculate the following statistic: 2 A d n d 2 n d Where: A d is the number of misses for a d bit shift n is the number of bits to be tested

32 25 I print the statistic and the number of misses for each shift count. The statistic is approximately standard normal. In using this test suite I found that the poker and autocorrelation tests were by far the most useful. If both of these tests indicated that the sequence was good then all the other tests would also generally show a good result. Conversely, if any of the four tests indicated a bad result I knew immediately that the generator was bad. 4.4 Certification In some situations TRNGs may need to be certified by a certification authority. Two possible certification schemes are the FIPS standard and the AIS31 standard The NIST FIPS Standard The US National Institute of Standards and Technology s (NIST) Security Requirements for Cryptographic Modules (FIPS-140-2) contains a section covering RNGs. The FIPS-140 standard was originally released as FIPS in January It was revised in May 2001 as FIPS and is applicable to all Federal agencies that use cryptographic-based security systems to protect sensitive information in computer and telecommunication systems.

33 26 The requirements for RNGs in this standard are currently undergoing significant change. In both the standard and the originally issued standard there were four power-up tests that the RNG had to pass. Depending on the level of certification the module had to either run these tests on start-up or on demand by the user of the module. The tests consisted of collecting 20,000 bits from the RNG and performing the following tests (from the FIPS standard): Monobit Test Count the number of ones in the 20,000-bit stream. Denote this quantity by X. The test is passed if 9,654 < X < 10,346. Poker Test - Divide the 20,000 bits into 5,000 four-bit segments. Count and store the number of occurrences of each of the 16 possible 4 bit values. Denote the number of each 4 bit value i where 0<=i<=15. Evaluate the following: f (i) as X 15 = i= 0 2 ( 16 / 5000) * [ f ( i)] 5000 The test is passed if 1.03<X<57.4 Runs Test A run is defined as a maximal sequence of consecutive bits of either all ones or all zeros, which is part of the 20,000-bit sample stream. The incidences of runs (for both consecutive zeros and consecutive ones) of all lengths (>=1) in the sample stream should be counted and stored.

34 27 The test is passed if the number of runs that occur (of lengths 1 through 6) is each within the corresponding interval. For the purpose of this test, runs of greater than 6 are considered to be of length 6. Length of Run Required Interval 1 2,267-2, ,079-1, Long Run Test A long run is defined to be a run of length 34 or more (of either zeros or ones). On a sample of 20,000 bits, the test is passed if there are NO long runs. The standard specifies one continuous test to be performed: Save the last 16 bits produced (or more if the RNG produces bits in blocks larger than 16) and compare them to the next 16 (or block of) bits produced. If they are the same, fail the test.

35 28 In December of 2002 NIST released revision 2 which completely removed the above power-up RNG tests from the standard. No explanation of the removal is given and no replacement tests are given. This leaves the standard with only the continuous test in effect. Annex C of the FIPS standard lists the PRNGs and TRNGs that are considered Approved by the standard. The approved PRNGs are: 1. Appendix 3.1 of the Digital Signature Standard (DSS) FIPS Appendix 3.2 of the Digital Signature Standard (DSS) FIPS Appendix A.2.4 of the American Bankers Association, Digital Signatures Using Reversible Public Key Cryptography of the Financial Services Industry (rdsa), ANSI X Annex A.4 of the American Bankers Association, Public Key Cryptography for the Financial Services Industry: The Elliptic Curve Digital Signature Algorithm (ECDSA), ANSI X There are no approved TRNGs.

36 The AIS 31 Proposed Standard The German Federal Office for Information Security (Bundesamt für Sicherheit in der Informationstechnik) (BSI) has published a certification scheme for PRNGs known as AIS 20. A second proposed standard for TRNGs is known as AIS 31 [1] [2]. Taken together these two standards set forth a methodology for evaluating deterministic and non-deterministic RNGs (i.e. PRNGs and TRNGs). The AIS 31 proposal defines two classes, P1 and P2. The classes describe the strength of the TRNG. A P1 class TRNG is only required to produce an output that is statistically inconspicuous. For a P2 class TRNG the prospects of success for systematic guessing of the external random numbers (relised through systematic exhaustion attacks) even if external random number sub-sequences are known should at best be negligibly higher than would be the case if the external random numbers had been generated by an ideal random number generator. The standard goes on to describe a number of sub-requirements for the two classes of TRNGs. One sub-requirement (property P1.d(ii) ) specifies a number of statistical tests that must be passed. Four of these tests (T1 T4) are taken from the FIPS tests described above. To these tests the AIS 31 standard adds the following:

37 30 (T0) A disjointness test (T5) An autocorrelation test (T6) A uniform distribution test (T7) A comparative test for multinomial distributions (T8) An entropy test The first six tests (T0-T5) are run on the output bits from the TRNG. The other three tests (T6 T8) are run on the raw bits from the internal noise (randomness) source. The rejection limits for the internal tests (T6 T8) are set with the knowledge that there may be a considerable bias present in the source bits. The sub-requirement of interest to my effort is for a P2 TRNG to constantly monitor the source of entropy and prevent the output of bits if the entropy source fails (property P2.d(x) ). This kind of total failure test is called a tot test in the standard. This might seem like an obvious requirement for a TRNG, however, it may be quite difficult to determine that the changing value coming from a supposedly noisy resistor is in fact the ripple from the power supply. When the on-line tests get triggered depends on what class the TRNG is to be certified to. It appears that as of this writing the AIS 31 standard is still in the proposal phase and has not been accepted as an official standard yet.

38 5. TRNG BACKGROUND 5.1 Sources of randomness A TRNG is always based on a physical source of randomness. Such sources of randomness include: Electrical Noise All resistors produce a small random signal when undriven. This signal is called Johnson noise and its amplitude is random and Gaussian. If this white noise signal is carefully extracted it can be used as good source of randomness. A second source of noise in resistors is known as Shot noise. A result of the quantum nature of a flowing current (i.e. the movement of the individual electrons), this noise is also Gaussian. A third kind of noise in a resistor is Flicker noise. This noise is based on the fluctuations in resistance in a resistor that has a current flowing through it. The noise has a 1/f spectrum (equal power per decade of frequency) [18]. 31

39 32 My TRNG relies on signal jitter which is a combination of these three kinds of electrical noise. Many commercial TRNGs use un-driven resistors (or diodes) to generate noise. The signal is then either sampled directly or used to drive a Voltage-Controlled Oscillator (VCO) which is sampled to produce a bit [31]. Quantum mechanical properties of photons In [20] a design for a RNG that uses a beam-splitter to cause individual photons to randomly choose one of two paths is presented. By using the path taken by the photon to drive a binary signal they produce random bits. Quantum theory predicts that the choice of path by the photon is truly random (and independent). Radioactivity The decay of unstable isotopes is believed to be a random event. A TRNG can be built by measuring the time between the detection of decay products. Usually, two intervals are compared; if the second one is longer, one value is output, if it is shorter, the other value is output. Human machine interactions By asking a human operator to type several tens of characters or move a mouse, a few random bits can be extracted from event timings.

40 33 Mechanical systems By examining timing information generated by reading a hard disk some random information based on air-flow disturbances in the drive can be extracted [10]. 5.2 Previous Work Electronically generating random bits has been attempted for many years. One early use of electronic RNGs was to generate five-bit paper tapes used to encrypt teletype traffic via a one-time pad scheme. In 1946 ATT was issued US patent for the device shown in Figure 4. The source of randomness was a large container of white and black balls. Figure 4 - ATT Patent

41 34 The source of randomness for my TRNG is two free-running ring oscillators. Using oscillators as a source of randomness has a long history. The bits for the RAND Corporation's 1955 book titled A Million Random Digits with 100,000 Normal Deviates were generated via two free-running oscillators [32]. New RNG designs are constantly being submitted for patent protection. The US Patent classification system contains the subclass ( ) for random number generators (which itself consists of six subclasses including a sub-subclass for oscillator controlled RNGs). Patents 4,810,975 and 4,855,690 issued in 1989 to Dallas Semiconductor Corp. for an oscillator controlled RNG are typical. In these patents one oscillator is used to drive a second voltage-controlled oscillator (VCO). A flip-flop is clocked by a computer to sample the output of the VCO and generate a random bit. Work on RNGs is on going. In June of 2003 patent 6,581,078 was issued for a RNG that uses a noise source to drive a PRNG. A very good discussion of building a two oscillator based RNG is contained in [15]. In this paper written for the CRYPTO 84 conference, Fairfield, Mortenson, and Coulthart discuss a design that uses two oscillators, one running much faster than the other. The slow oscillator uses a D-type flip flop to sample the higher frequency oscillator. They show that if the jitter in the slow oscillator signal is sufficient, the output of the RNG will

42 35 have very little bit-to-bit correlation (their signal does not have sufficient jitter so they use a correlation correction scheme). Another discussion of dual oscillator based RNGs is contained in [14]. In their paper presented at the 2003 Cryptographic Hardware and Embedded Systems conference (CHES 2003) Epstein et al. use two ring oscillators which they can switch together to form a bi-stable. When the transition to the bi-stable configuration occurs the inputs are sometimes in the process of changing thus producing a metastable state. A series of outputs are XORed to form a random bit. A good discussion of how to mathematically model and then construct a noise based TRNG for cryptographic use is contained in [5]. A discussion of how to simulate oscillator-based RNGs is contained in [30]. Two recent papers detail the difficulties of building RNGs in FPGAs. One, presented at the IEEE Symposium on FPGA-Based Custom Computing Machines (FCCM 2003) by Tosi et. al. [38] describes a low bit rate TRNG implemented in an FPGA that uses an external RC circuit as a source of randomness. The output bits from this TRNG are used to re-seed a PRNG that is also implemented in the FPGA. The second paper, by Fischer and Drutarovský, was presented at the Cryptographic Hardware and Embedded Systems conference (CHES 2002). I discussed the Fischer and Drutarovský design in Chapter 3.

43 The Intel RNG The Intel RNG, built into all 8xx chipsets, is a good example of the resistor noise technique. The Intel RNG samples the noise across an undriven resistor by amplifying it and using it to drive a voltage controlled oscillator. A second high frequency oscillator latches the VCO output and the result is fed to a von Neumann corrector. One potential significant problem with this noise amplification technique is correlated local pseudorandom noise sources (e.g. power supply ripple). The Intel RNG reduces the effect of these sources by subtracting the signals from two nearby similarly undriven resistors. Intel commissioned an analysis of their RNG from Cryptography Research Inc (CR) [21]. CR was given access to measurements of internal signals from the RNG. They were also able to gather output from the RNG under a variety of environmental conditions. The output was analyzed using a number of statistical tests including the DIEHARD suite. It is unclear if Intel plans to continue to embed a RNG in their chipsets. The following statement precedes the current documentation about the RNG on their web site: End of Interactive Support Announcement: These products and tools are no longer being manufactured by Intel. Additionally, Intel no longer provides any interactive support for these via telephone or ,

44 37 nor will Intel provide any future software updates to support new operating systems or improve compatibility with third party devices and software products. 5.4 The VIA C3 Nehemiah RNG VIA Technologies Inc. is a developer of core logic chipsets (thus they are a competitor of Intel). They have developed a RNG of their own and also had it evaluated by Cryptography Research Inc [9]. The VIA design uses four VCOs (two of which control a third) to drive a sampling flip-flop and produce a stream of random bits. The CR analysis claims that the output of this arrangement has considerable bias and that VIA needed a von-neumann Corrector to reduce the bias to an acceptable level.

45 6. MY TRNG DESIGN 6.1 Overview My design consists of two independent and identically configured ring oscillators, a sampling circuit, and a control circuit. Ring Oscillator Ring Oscillator Clk0 Clk1 Sampler RandBitOut BitReady ReadAck Reset Sample Enable Controller Figure 5 - TRNG Design The two ring oscillators each supply a stream of pulses to the sampler unit. The frequency of the two clock signals is chosen to be close but not identical. The sampler unit uses one clock signal to sample the other clock signal. The stream of samples consists of a run of ones and a gap of zeros. The length of this run and gap is counted modulo 2 and output as a random bit. 38

46 The Ring Oscillator ClkOut D Q D Q 0 G 0 G Figure 6 - The ring oscillator circuit The output from the ring oscillator is a stream of regular pulses. The ring oscillator consists of several gates and an inverter configured serially to feed back on itself. The sum of the propagation delays generator determines the nominal output frequency p i f through the various elements of the of the circuit. f = 2 1 p i I found this basic configuration for a ring oscillator to be quite stable when I took care to have the end-to-end propagation delay of the circuit well above the inertial delays of the individual elements.

47 40 I tested several different configurations for my oscillator. The output bit rate of my TRNG is directly related to the frequency of the ring oscillators, thus a high oscillator frequency was desirable. A target frequency of 150MHz would allow me to easily create counters and other control logic to test the design without the need for substantial logic optimization while still providing reasonable output bit rates. The ring oscillator propagation delay implied by 150MHz is: 1 2*150,000,000 = 3.333ns The propagation delay through the average gate or latch is approximately 0.4 ns and the wire delay from the output of the CLB back to the input of the CLB is about 1.0 ns. Thus I quickly settled on the design shown in Figure 6: FeedBack1 ClkOut A4 A3 A2 A1 LUT D=A1 D D G Q Init A4 A3 A2 A1 LUT D=~A 1 D D G Q Init ClkEnable ClkReset FeedBack0 Figure 7 - CLB Layout

48 41 This configuration exactly fit in one Virtex CLB slice (Figure 7). The output of both latches (configured with negative logic so a zero at the Gate results in the latch being transparent) is routed externally from the output of the CLB back to its inputs. The output of the generator is taken from the output of the buffer. Note that the lower LUT implements an inverter while the upper LUT is just a buffer. I found the end-to-end propagation delay through this particular circuit to be approximately 3.7 ns, which resulted in an output frequency of 130MHz. I note that the individual components of the circuit each experience a change of input every 3.7 ns, which is almost an order of magnitude greater then their inertial delays. In my testing, I found these ring oscillators to be very reliable. CLB ring oscillators, such as the one above, have an intrinsic natural difference in speed. These differences are due to small variations in the physical construction of each CLB. I expand later in this thesis on the effects of placement of the ring oscillator CLBs on the FPGA and the effects of temperature on the speed of the ring oscillator. 6.3 The Sampler The sampler circuit extracts the jitter contained in the signals from the two ring oscillators.

49 42 Clk0 D Q S0 1 D Q BitReady Clk1 CE Init ReadAck D Q C0 D Q RandOut Init CE R0 S0 E0 From/To Control Figure 8 - The sampler circuit Clk1 Clk0 S0 C0 RandOut 0 1 Figure 9 - Timing diagram for the sampler circuit As shown in Figure 8, at the input of the sampler circuit a D type flip flop uses the CLK1 signal to sample the CLK0 signal. The output of the sampling process (without jitter) is illustrated in Figure 9. The signal S0 will be high so long as the rising edges of CLK1 occur during the high portion of CLK0. Once CLK1's rising edge starts sampling the low portion of CLK0 the S0 signal will transition to a '0'.

50 43 Figure 10 shows what happens as the sampling point moves through a signal with jitter. The jitter in the CLK0 signal will be captured and expressed as a change in the cycle length of the S0 signal. In my sampler circuit I set up a one-bit counter to count cycles in the CLK1 signal (signal C0). By using the S0 signal to latch the value of the C0 counter I can convert the Least Significant Bit of the length of the S0 signal to a single random bit (RandOut). The S0 signal is also used to notify the user of the TRNG that a new random bit is ready. One key advantage to using this technique is that it captures the essential random element (the cycle length uncertainty) and very simply presents it as a single random bit without having to have a priori knowledge of the frequencies involved. Clk1 Clk0 S0 Uncertain cycle C0 RandOut 0 1 Figure 10 - Sampler circuit's behavior with jitter 6.4 The Control Circuits The description in the previous section only considered the jitter on the CLK0 signal. Since my ring oscillators are identically constructed it is reasonable to assume that they

51 44 have similar amounts of jitter. In cases where the difference in the cycle lengths of the two clock signals is very small it is possible for the S0 signal to transition several times before settling down to a stable value. Without the control circuit, the sizes of these small S0 cycles will be counted and presented as (very correlated) random bits. The output of the control circuit drives the Clock Enable (CE) inputs of the control and output flip-flops. The control circuit disables these devices immediately after a random bit is latched into the output flip-flop. The control enables the clock inputs on these two devices only after it has counted a preset number of CLK1 cycles that have sampled the low portion of the CLK0 signal. In this way it forces the sampler circuit to ignore the short S0 cycles that occur on both the rising and falling edge of S0. The control circuit also resets the one bit counter (via signal R0) after each random bit is latched. I do this to eliminate any correlation between successive bits. An important secondary benefit of the control circuit is that it prevents any output from the TRNG if the difference between the cycle lengths of the two clock generators is too great. If the difference in the cycle time of CLK0 and CLK1 is greater then the width of the jitter zone then some S0 cycles will not contain a sample that includes jitter. Larger cycle time differences also produce fewer samples of CLK0. The control circuit will never enable the output flip-flops if there are too few cycles of '0' in the S0 signal. Detecting the failure of the internal source of randomness is a required function of a TRNG.

52 Ring Oscillator Issues One result of my research was the discovery of a wide variation in the intrinsic speed of ring oscillators in an FPGA. I found a 7% difference between the normalized speeds of the slowest CLB and the fastest CLB. S26 S21 S16 S11 S S1 Figure 11 - Normalized CLB speeds for rows 1-30 In Figure 11 each color represents a range of possible speeds for a ring oscillator embedded in a CLB. Loading two ring oscillators into an FPGA and running them for 20 seconds determined the ring oscillator speeds. By keeping one of the oscillators fixed to the same CLB location from run-to-run I was able to normalize the speeds for all the CLBs measured. Several things become evident from this graph:

53 46 1. CLBs on one side of the FPGA are faster then CLBs on the other side. I think the most likely explanation for this phenomenon is that the FPGA s heat sink has air flowing across it and one side is cooler than the other. 2. There are clearly internal structures that cause CLBs near the structures to run at different speeds. 3. Variances in the construction materials and in the way the material is applied during manufacturing may cause CLBs to run at different speeds. My randomness extraction technique requires a pair of ring oscillators with closely matched frequencies. The frequency of a ring oscillator tends to wander as the temperature of the chip varies 2. Even on a mostly empty FPGA there is a several second period in which the speed of the ring oscillator decreases as the area around the oscillator CLB heats up. It is important that frequency of the two oscillators does not wander apart due to temperature differences on the chip. For this reason I found that it was important to place the two oscillators close to each other. 2 Precisely this effect is used to measure the temperature of individual CLBs in [25]