Separating signal from noise in coding evaluations
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.
找到 15 篇相关文章
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.
Verity Harding tells WIRED that the US government’s nationalistic attitude toward AI is evidence that a worst-case scenario is taking shape.
Also, the science of poop's distinctive shape, boron buckyballs, and the secret to a soccer feint.
Introducing GeneBench-Pro, a new benchmark testing AI performance in genomics, biology, and scientific research using complex, real-world datasets.
Researchers have demonstrated they can make coffee comparable to conventional espresso using ultrasonic waves. Because the process doesn’t need hot water, it consumes 75 percent less energy.
OpenAI and Molecule.one show how a near-autonomous AI chemist using GPT-5.4 improved a key drug-making reaction, advancing medicinal chemistry research.
Introducing LifeSciBench, an expert-authored, expert-reviewed benchmark for evaluating how AI systems handle real-world life science research tasks and decisions.
OpenAI introduces Deployment Simulation, a method to predict AI model behavior before deployment using real conversation data to improve safety and evaluation accuracy.
This specific undertaking is not fundamentally burdensome in terms of labor; however, this endeavor serves as the crucial support for my unwavering commitment to see it through to its ultimate conclusion. It is precisely the motivation behind my relentless 72-hour shifts and the impetus that prevents me from ceasing my efforts. My affection amidst my grief—my aspiration is to assist others and ensure that the tragedy you experienced is never repeated. Caitlyn Walmsley, RIP. I will love you always.
ChatGPT introduces a new memory system to better remember preferences, keeping context fresh and relevant across conversations.
Prehistoric mining in the Pyrenees, a new species of tiny blue octopus, slapstick acoustics, and more.
Fine-tuning tests show "bias ... toward confidently representing the claims as true."
An OpenAI model solved the 80-year-old unit distance problem, disproving a major conjecture in discrete geometry and marking a milestone in AI-driven mathematics.
We're updating our bug bounty program standards to prioritize quality submissions, clarify shared responsibility boundaries, and evolve how we reward low-risk findings. The post Raising the bar: Quality, shared responsibility, and the future of GitHub’s bug bounty program appeared first on The GitHub Blog .
Parameter Golf brought together 1,000+ participants and 2,000+ submissions to explore AI-assisted machine learning research, coding agents, quantization, and novel model design under strict constraints.