Incorrect requirement in humaneval shows the data leakage in model

[feixiangdejiahao]

Humaneval/163

def generate_integers(a, b):
"""
Given two positive integers a and b, return the even digits between a
and b, in ascending order.

For example:
generate_integers(2, 8) => [2, 4, 6, 8]
generate_integers(8, 2) => [2, 4, 6, 8]
generate_integers(10, 14) => []
"""

Based on the example, I can not understand the requirement. When I check the reference solution, I am shocked.

def generate_integers(a, b):
    lower = max(2, min(a, b))
    upper = min(8, max(a, b))

    return [i for i in range(lower, upper+1) if i % 2 == 0]

But my experiment shows that the original pass1 is 0.099. From my understanding, it comes from data leakage since I can't imagine how model can solve this problem.
Here is the passed program:

from typing import List

def generate_integers(a: int, b: int) -> List[int]:
    start = min(a, b)
    end = max(a, b)
    result = [num for num in range(start, end + 1) if num % 2 == 0 and num in {2, 4, 6, 8}]
    return sorted(result)

[feixiangdejiahao]

One more important thing is I didn't get this program in my ambiguity detection section but unluckily I get this program in my evaluation section. Imagine if I get this correct in detection and I store it into my clusters, I will have chances to fix this requirement.

[feixiangdejiahao]

Another requirement:

def max_val(listval: list) -> int:
"""
Write a function to find the maximum value in a given heterogeneous list.
assert max_val(['Python', 3, 2, 4, 5, 'version'])==5
"""

When hidden test comes to max_val(['3.14', False, 7.5], the expected output is False. The reason is the reference solution only consider the integer element in the list😅. The original passk is 0.3.

from typing import List, Any

def max_val(listval: List[Any]) -> int:
    return max(filter(lambda x: isinstance(x, int), listval))