GATE Electronics (EC): Communications (random processes, AM and FM, information theory, digital modulation, PCM and coding), previous-year questions solved, part 2

Part 2 of 3 of the Communications hub: 8 more official GATE EC questions on information theory and channel capacity, each solved step by step and checked against the official answer key. Part 1 covers random processes and noise, analog modulation and receivers, digital modulation and detection and pulse code modulation and error-control coding.

Questions from official GATE EC papers (IIT Kharagpur 2022, IIT Kanpur 2023, IISc Bengaluru 2024, IIT Roorkee 2025, IIT Guwahati 2026); solutions by Skill-Lync. · Updated 17 Nov 2026 · 8 of 31 questions on this page

GATE 2022 · EC · Q.29MSQ · 1 markInformation theory and channel capacity

Entropy H(X) of a discrete variable with K values: which statements are necessarily true?

Let denote the entropy of a discrete random variable taking possible distinct real values. Which of the following statements is/are necessarily true?
  1. (A) bits
  2. (B)
  3. (C)
  4. (D)

Answer: (A) and (B) and (D) Official key: A,B,D

  1. Entropy depends only on the probabilities of the outcomes, not on the values the outcomes carry. If is one to one, then has exactly the same list of probabilities as , so .
  2. (A) The entropy of a source with outcomes is largest when all are equally likely, and then bits. So always. True.
  3. (B) is one to one on the real line, so and the inequality holds with equality. True.
  4. (C) is not one to one: and land on the same value. Merging outcomes can only lose information, so , and the inequality as written () holds only when no merging happens. Counter-example: with probability each gives bit but always, so . Not necessarily true.
  5. (D) is one to one on the real line (strictly increasing), so . True.
  6. Answer: (A), (B) and (D).

Trap: option (C) looks the same as (B) and (D) because it is also a function of , but a many-to-one function can lose entropy. The code drew 4000 random discrete variables (2 to 8 distinct integer values between and , random probabilities): (A), (B) and (D) were never violated (equality for (B) and (D), as the maps are one to one), while (C) failed 2379 times. It also confirmed the counter-example against .

Where this shows up in industry

This is the data-processing idea behind lossy front ends: any many-to-one step (quantising, clipping, taking a magnitude) cannot raise the information in a signal, while a reversible step such as scaling or a monotonic companding law keeps it. Source coders such as Huffman and arithmetic coders use the entropy as the floor on bits per symbol.

GATE 2022 · EC · Q.61NAT · 2 marksInformation theory and channel capacity

Capacity of a 3-input 3-output channel with crossover alpha: which alpha in [0.25, 1] maximises it?

The transition diagram of a discrete memoryless channel with three input symbols and three output symbols is shown in the figure. The transition probabilities are as marked.

The parameter lies in the interval . The value of for which the capacity of this channel is maximized, is ________ (rounded off to two decimal places).
Discrete memoryless channel diagram with inputs a, b, c on the left and outputs a, b, c on the right. Each input goes straight across to the output with the same letter with probability 1 minus alpha. Input a goes to output b with probability alpha, input b goes to output c with probability alpha, and input c goes to output a with probability alpha1−α1−α1−ααααaabbcc

Answer: Official key: 1.00 to 1.00

  1. From the diagram: input goes to with and to with ; input goes to with and to with ; input goes to with and to with .
  2. Every row of the transition matrix is a permutation of , and every column too (each output is reached by one straight and one crossing edge). The channel is symmetric, so the uniform input is optimal and is the same for every input: .
  3. With uniform input the output is uniform too, so and
  4. is largest where is smallest. On , falls steadily from bits ( peaks at ) down to , so the minimum is at the end of the interval.
  5. At every input is sent to a different output with certainty (a noiseless relabelling), so bits per use. Answer: .

Trap: answering because it is the special value of the binary entropy function. That is where the capacity is smallest, not largest: bits, against and . The code maximised over a grid on the input simplex and with the Blahut-Arimoto iteration for ; both agreed with and the maximum was at .

Where this shows up in industry

The same bookkeeping, capacity equals output entropy minus the noise entropy per input, is how link adaptation reasons about discrete channels: a channel that maps inputs to distinct outputs with certainty carries the full bits, and any crossover probability subtracts -like terms. Shannon capacity is the ceiling that modem and coding designs are measured against.

GATE 2023 · EC · Q.65NAT · 2 marksInformation theory and channel capacity

Average number of yes/no questions to identify one of 8 symbols with probabilities 1/2, 1/4, ..., 1/128, 1/128

The frequency of occurrence of 8 symbols (a-h) is shown in the table below. A symbol is chosen and it is determined by asking a series of “yes/no” questions which are assumed to be truthfully answered. The average number of questions when asked in the most efficient sequence, to determine the chosen symbol, is _______ (rounded off to two decimal places).
Symbolsabcdefgh
Frequency of occurrence

Answer: Official key: 1.97 to 1.99

  1. A series of yes/no questions that always ends at one symbol is a binary code tree: each answer is one bit, and the number of questions for a symbol is its code length. The most efficient sequence is the one with the smallest average length, a Huffman code.
  2. Merge the two least likely symbols repeatedly: ; with this gives ; with , ; with , ; with , ; with , ; with , .
  3. Code lengths: . The questions are 'is it ?', then 'is it ?', and so on down the list, ending with one last question to separate from .
  4. .
  5. Check with the entropy: every probability is a power of , so is a whole number and equals the length used. bits, the same value, so no scheme can do better. Answer: .

Trap: answering by treating the 8 symbols as equally likely ( questions each, a plain binary search), which ignores the frequencies. The code ran a Huffman construction (lengths [1, 2, 3, 4, 5, 6, 7, 7], average 1.984375) and, as an independent check, an exhaustive dynamic programme over every possible yes/no question tree on the 8 symbols, which gave 1.984375. The entropy is 1.984375 bits. The equal-probability answer is outside the official range to .

Where this shows up in industry

Huffman coding is the entropy-coding step inside DEFLATE (used by gzip, ZIP and PNG) and in JPEG. The source entropy is the floor on the average code length, and the codes reach it exactly only when all probabilities are powers of one half, as here.

GATE 2024 · EC · Q.34NAT · 1 markInformation theory and channel capacity

Maximum achievable entropy of a source with an alphabet of 16 symbols

A source transmits symbols from an alphabet of size 16. The value of maximum achievable entropy (in bits) is _______ .

Answer: Official key: 4 to 4

  1. Entropy is with .
  2. It is largest when all symbols are equally likely, . (Lagrange multiplier: gives the same for every .)
  3. bits per symbol.

Trap: answering (the number of symbols) or in nats (). The code summed over random distributions on 16 symbols (none exceeded ), ran an exponentiated-gradient ascent from 20 random starts (all converged to ), and checked that one-hot, two-equal and eight-equal distributions give , and bits.

Where this shows up in industry

Source coding sets its goal from this bound: a 16-level quantiser output or a 16-QAM symbol stream can carry at most bits per symbol, and real data only reaches that when it is compressed so the symbols look uniform. Entropy coders such as Huffman and arithmetic coders aim to approach the source entropy.

GATE 2025 · EC · Q.15MCQ · 1 markInformation theory and channel capacity

AWGN channel with fixed Pav and N0: which plot shows capacity C against bandwidth W?

Consider an additive white Gaussian noise (AWGN) channel with bandwidth and noise power spectral density . Let denote the average transmit power constraint. Which one of the following plots illustrates the dependence of the channel capacity on the bandwidth (keeping and fixed)?
  1. (A)Option A: C rises steeply from the origin, flattens and levels off at a constant value as W growsCW0
  2. (B)Option B: C is a straight line through the origin, growing without limit as W growsCW0
  3. (C)Option C: C starts slowly and then rises faster and faster as W growsCW0
  4. (D)Option D: C rises to a maximum at a finite W and then falls as W grows furtherCW0

Answer: (A) Official key: A

  1. Noise power in bandwidth is (density on both and over a band gives ). Shannon-Hartley: .
  2. Small : is large, so . rises steeply because each extra hertz carries many bits.
  3. for all (more bandwidth never hurts) and (each added hertz helps less than the one before). So the curve rises and bends over; it cannot have a maximum (D) or curve upward (C).
  4. Large : use with : , a finite limit. Extra bandwidth also lets in extra noise, so saturates and does not grow like the straight line (B).
  5. The curve that rises steeply, bends over and levels off at is option (A).

Trap: option (B), a straight line, is the picture many students carry from 'capacity is proportional to bandwidth', which holds only when the signal-to-noise ratio is held fixed. Here the noise power grows with . The code evaluated for on log-spaced points from to : increases at every step, the second difference is negative at every point, and against . falls from 9.97 at to 1.4e-05 at , so it is not a straight line, and there is no interior maximum. Only (A) passes all tests.

Where this shows up in industry

This saturation is why a link budget cannot be fixed by buying more spectrum alone once the link is noise-limited at wide bandwidth: the ceiling is set by received power per unit noise density. Link-adaptation software in LTE and 5G NR picks modulation and code rate against a capacity bound of this form, and wideband sensors trade bandwidth against the same way.

GATE 2025 · EC · Q.54MSQ · 2 marksInformation theory and channel capacity

Entropy g(alpha) of X in {-1, 0, 1} with P(+-1) = alpha, P(0) = 1 - 2 alpha: which comparisons hold?

The random variable takes values in with probabilities and , where . Let denote the entropy of (in bits), parameterized by . Which of the following statements is/are TRUE?
  1. (A)
  2. (B)
  3. (C)
  4. (D)

Answer: (B) and (C) Official key: B;C

  1. .
  2. Entropy is largest when the three outcomes are equally likely: , so , where bits. Entropy is concave in the probabilities, so rises up to and falls after it.
  3. Evaluate: bits. bits. bits.
  4. , so (B) is true and (A) is false. , so (C) is true and (D) is false.
  5. Correct options: (B) and (C). Distances from the peak at settle it quickly: is away, is away and is away.

Trap: option (A) comes from thinking that a larger means a larger entropy. It does, only up to . The code summed over the three outcomes and got , , ; a sweep of over points found the maximum at with (). Testing the four comparisons gave exactly (B) and (C). Option (D) is wrong because is farther from than is, on the same side.

Where this shows up in industry

Entropy is the floor on the average number of bits per symbol that any lossless source code can reach, so the peak at equal probabilities explains why a compressor gains nothing on data that already looks uniformly random. Three-level symbols with an unlikely zero or a likely zero, as in line codes like AMI and 4B3T, are costed this way.

GATE 2025 · EC · Q.60NAT · 2 marksInformation theory and channel capacity

Bernoulli X, Y with joint pmf 0.06, 0.14, 0.24, 0.56: mutual information I(X;Y)

and are Bernoulli random variables taking values in . The joint probability mass function of the random variables is given by: , , , . The mutual information is _________ (rounded off to two decimal places).

Answer: Official key: 0.0 to 0.0

  1. Marginals: and . and .
  2. Compare the joint probabilities with the product of marginals: , , , . All four match, so and are independent.
  3. bits.

Trap: a student who computes the entropies and stops, or who does not check the marginals, ends up with a non-zero number: , and bits, and none of these is the answer. The code summed over the four cells and got , and . The KL divergence of the joint from the product of marginals is , and the determinant confirms that the joint matrix has rank , which means independence.

Where this shows up in industry

Zero mutual information means the received value tells you nothing about what was sent, which is the state of a channel that is pure noise, for example a receiver whose reference oscillator has lost lock. Link-layer designers estimate mutual information between transmitted and received symbols to see how much of a channel's capacity is actually in use.

GATE 2026 · EC · Q.55MSQ · 2 marksInformation theory and channel capacity

Shannon capacity C versus bandwidth B = 1 MHz with kT = -174 dBm/Hz and received power S = -80 dBm

Let the relevant bandwidth () of a digital communication system be MHz and dBm/Hz, where is Boltzmann's constant and is equivalent noise temperature of the receiver. The power () of signal received through an additive Gaussian channel is dBm.
Which of the following options is/are TRUE about Shannon capacity () of the channel?
  1. (A)
  2. (B)
  3. (C)
  4. (D)

Answer: (C) Official key: C

  1. Noise power in the band: . In dB: .
  2. Signal-to-noise ratio: , that is .
  3. Shannon-Hartley: , about Mbit/s.
  4. Test the options. needs ( dB); we have dB, so (C) is true. would need , would need , and would need , so (A), (B) and (D) are false. The answer is (C).

Trap: option (D), , is what you get if you divide the dBm numbers as if they were linear ratios: gives . Decibels must be subtracted, then converted to a power ratio. The code converted dBm to watts, got , computed , and then maximised the mutual information of a numerically integrated Gaussian channel (differential entropies by trapezoid integration, real samples per second): , matching. Testing each option on that value left only (C).

Where this shows up in industry

This is the link-budget calculation behind every receiver spec: thermal noise of -174 dBm/Hz plus 10 log10 of the bandwidth gives the noise floor, and the margin of received power above it sets the best bit rate any modulation and coding scheme can reach. Link adaptation in Wi-Fi and cellular radios picks a modulation and code rate that sits a few dB below this Shannon ceiling.

From GATE prep to a telecom career

Talk to a counsellor.

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.

Talk to a career counsellor

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.