This week InfoWorld unveiled their annual list of "the leading open source projects for software development, cloud computing, big data, and machine learning." [E]ven as we grapple with the likes of microservice architecture, distributed data processing frameworks, deep neural networks, and "dapps," we remain steadfast in our commitment to bring you -- this year and every year -- the best that open source has to offer.

In this year's edition, you'll find our picks for the best open source software development tools, cloud computing platforms, databases and data analytics tools, and machine learning and deep learning libraries. From Kubernetes and Docker to TensorFlow and PyTorch (49 projects in all), these are the projects that are ushering in the next stage of enterprise computing.
An anonymous reader writes: Their choices for the best open source software for software development include .NET Core, Microsoft's Visual Studio Code, and Jenkins, as well as programming languages like Kotlin, Julia, and Rust. ("By now it's something of a cliche to talk about Rust as the next step beyond C and C++. So be it...") And their final award for best open source development software went, surprisingly, to Vanilla JS.

"Some clever wag created a website that promises that the Vanilla JS library will be the smallest JS framework you'll ever use and then delivers a zip file with zero bytes of code along with the suggestion that you should just use the built-in function calls in JavaScript to manipulate the DOM."

schwit1 shared an article from the New York Times: When investigative journalist Julia Angwin worked for ProPublica, the nonprofit news organization became known as "Big Tech's scariest watchdog." By partnering with programmers and data scientists, Angwin pioneered the work of studying Big Tech's algorithms -- the secret codes that have an enormous effect on everyday American life... Now, with a $20 million gift from Craigslist founder Craig Newmark, she and her partner at ProPublica, data journalist Jeff Larson, are starting the Markup, a news site dedicated to investigating technology and its effect on society. Sue Gardner, former head of the Wikimedia Foundation, which hosts Wikipedia, will be the Markup's executive director. Angwin and Larson said that they would hire two dozen journalists for its New York office and that stories would start going up on the website in early 2019...

Angwin, who was part of a Wall Street Journal team that won a Pulitzer Prize in 2003 for coverage of corporate corruption, said the newsroom would be guided by the scientific method, and each story would begin with a hypothesis... At the Markup, journalists will be partnered with a programmer from a story's inception until its completion. "To investigate technology, you need to understand technology," said Angwin, 47... Newmark, who splits his time between San Francisco and New York, has for years kept a low profile. But he worries about what he sees as a lack of self-reflection among engineers. "Sometimes it takes an engineer a while to understand that we need help, then we get that help, and then we do a lot better," Newmark said. "We need the help that only investigative reporting with good data science can provide...."

Engineers being surprised by the tools they have made is, to the Markup team, part of the problem. "Part of the premise of the Markup is the level of understanding technology and its effects is very, very low, and we would all benefit from a broader understanding," Gardner said. "And I would include people who work for the companies."
Larson laments a world where programs handle crucial decisions, and "once they go into production, there's no oversight..." Or, as he says earlier, "Increasingly, algorithms are used as shorthand for passing the buck." The Markup's site promises " a nonpartisan, nonprofit newsroom" offering independent analysis of how technology is re-shaping everything from what we believe to "who goes to prison versus who remains free." The site's donations page adds that "We strive for fairness and independence and for us, the best way to achieve that is to operate without ads or a paywall."

Angwin tells Recode that she grew up in Steve Jobs' neighborhood in Palo Alto, and in a long interview reveals that she learned to program in a fifth grade class that a public-spirited Steve Jobs funded. Now the Times points out that the Markup "will release all its stories under a creative commons license so other organizations can republish them, as ProPublica does."

After breaking into the top three most popular programming languages for the first time this month, behind C and Java, Python has also won the hearts of hackers and web nasties, according to attack statistics published this week by web security biz Imperva. From a report: The company says more than a third of daily attacks against sites the company protects come from a malicious or legitimate tool coded in Python. Imperva says that around 77 percent of all the sites the company protects, have been attacked by at least one Python-based tool. Furthermore, when the company looked at the list of tools that hackers used for their attacks, more than a quarter were coded in Python, by far the attackers' favorite tool. "Hackers, like developers, enjoy Python's advantages which makes it a popular hacking tool," the Imperva team says.

Xbox's Phil Spencer announced today that mouse and keyboard support is coming to Xbox One. Crytek's Warface will be the first game to test the feature when it becomes available in October via the Xbox Insider Program. IGN reports: The idea behind mouse and keyboard support will be as a tool for developers, so they can choose how they want the control style integrated, if they want it integrated at all. "If you're a dominant FPS player right now on controller and you're worried that all the sudden you're gonna get swamped because a bunch of mouse and keyboard players are gonna get flooded into your game, that's not what we're doing," said Spencer. "We're putting choice into the hands of the developers about the games that they want to bring." We can expect to hear more about mouse and keyboard support during the XO18 fan event in Mexico City on November 10.

Despite its advanced age, Cobol is still the most prevalent programming language in the financial-services industry world-wide. Software programmed in Cobol powers millions of banking transactions every day and underpins critical computer mainframes. WSJ: And Cobol isn't going away anytime soon. Banks and other companies have come to the uncomfortable realization that ripping out old mainframes is pricey and complicated. Transitioning to new systems is likely to take years, and besides, a lot of the older tech works just fine. The problem is that Cobol isn't popular with new programmers. So, with a generation of Cobol specialists retiring, there is a continuing hunt to find a new generation of programmers to service this technology. In Texas, Mr. Hinshaw's (an anecdote in the story) company, the Cobol Cowboys, a group of mostly older programmers, is training U.S. military veterans in the programming language. Accenture is coaching hundreds of Cobol programmers every year in India and the Philippines to work at banks. In Malaysia, one consultancy that provides engineers versed in Cobol for its clients, iTAc MSC Outsourcing, has adopted the slogan "Keeping the Dinosaurs Alive." A host of companies offer online courses in Cobol in places like South Africa, India and Bangladesh. Developing economies are key technology-outsourcing centers for banks. Further reading: Major Banks and Parts of Federal Gov't Still Rely On COBOL, Now Scrambling To Find IT 'Cowboys' To Keep Things Afloat.

An anonymous reader quotes a report from IEEE Spectrum: [A] software remedy can't solve Subaru's issue with 293 of its 2019 Ascent SUVs. All 293 of the SUVs that were built in July will be scrapped because they are missing critical spot welds. According to Subaru's recall notice [PDF] filed with the U.S. National Highway Transportation Safety Administration, the welding robots at the Subaru Indiana Automotive plant in Lafayette, Ind., were improperly coded, which meant the robots omitted the spot welds required on the Ascents' B-pillar. Consumer Reports states that the B-pillar holds the second-row door hinges. As a result, the strength of the affected Ascents' bodies may be reduced, increasing the possibility of passenger injuries in a crash. Subaru indicated in the recall that "there is no physical remedy available; therefore, any vehicles found with missing welds will be destroyed." Luckily, only nine Ascents had been sold, and those customers are going to receive new vehicles. The rest were on dealer lots or in transit.

An anonymous reader quotes a report from the BBC: Computer historians have staged a re-enactment of World War Two code-cracking at Bletchley Park. A replica code-breaking computer called a Bombe was used to decipher a message scrambled by an Enigma machine. Held at the National Museum of Computing (TNMOC), the event honored Polish help with wartime code-cracking. Enigma machines were used extensively by the German army and navy during World War Two. This prompted a massive effort by the Allies to crack the complex method they employed to scramble messages. That effort was co-ordinated via Bletchley Park and resulted in the creation of the Bombe, said Paul Kellar who helps to keep a replica machine running at the museum. Renowned mathematician Alan Turing was instrumental in the creation of the original Bombe.

For its re-enactment, TNMOC recruited a team of 12 and used a replica Bombe that, until recently, had been on display at the Bletchley Park museum next door. The electro-mechanical Bombe was designed to discover which settings the German Enigma operators used to scramble their messages. As with World War Two messages, the TNMOC team began with a hint or educated guess about the content of the message, known as a "crib," which was used to set up the Bombe. The machine then cranked through the millions of possible combinations until it came to a "good stop," said Mr Kellar. This indicated that the Bombe had found key portions of the settings used to turn readable German into gobbledygook. After that, said Mr Kellar, it was just a matter of time before the 12-strong team cracked the message.

LLVM release manager Hans Wennborg announced Wednesday the official availability of LLVM 7.0 compiler stack as well as associated sub-projects including the Clang 7.0 C/C++ compiler front-end, Compiler-RT, libc++, libunwind, LLDB, and others. From a report: There is a lot of LLVM improvements ranging from CPU improvements for many different architectures, Vega 20 support among many other AMDGPU back-end improvements, the new machine code analyzer utility, and more. The notable Clang C/C++ compiler has picked up support for function multi-versioning (FMV), initial OpenCL C++ support, and many other additions. See my LLVM 7.0 / Clang 7.0 feature overview for more details on the changes with this six-month open-source compiler stack update. Wennborg's release statement can be read on the llvm-announce list.

Here's an analysis of the Linux kernel repository that attempts to find some fresh numbers on the current kernel development trends. He writes: The kernel repository is at 782,487 commits in total from around 19.009 different authors. The repository is made up of 61,725 files and from there around 25,584,633 lines -- keep in mind there is also documentation, Kconfig build files, various helpers/utilities, etc. So far this year there has been 49,647 commits that added 2,229,836 lines of code while dropping 2,004,759 lines of code. Or a net gain of just 225,077 lines. Keep in mind there was the removal of some old CPU architectures and other code removed in kernels this year so while a lot of new functionality was added, thanks to some cleaning, the kernel didn't bloat up as much as one might have otherwise expected. In 2017 there were 80,603 commits with 3,911,061 additions and 1,385,507 deletions. Given just over one quarter to go, on a commit and line count 2018 might come in lower than the two previous years.

Linus Torvalds remains the most frequent committer at just over 3% while the other top contributions to the kernel this year are the usual suspects: David S. Miller, Arnd Bergmann, Colin Ian King, Chris Wilson, and Christoph Hellwig. So far in 2018 there were commits from 3,320 different email addresses. This is actually significantly lower than in previous years.

An anonymous reader writes: On Apple's iPhone day this week, Google announced it is killing Inbox by Gmail. But that's not the only service that day the company confirmed it is shutting down: The mobile app development tool Fabric is also going away. Firebase, Google's mobile and web application development platform, is swallowing Fabric and all its features. Incidentally, both Fabric and Firebase were once separate companies: Google acquired Fabric from Twitter in January 2017 and bought Firebase in October 2014. Now the company is merging the former into the latter, ending support for Fabric in mid-2019.

Facebook is trying to speed up the time it takes to roll out new software updates and debug any issues in them with a new tool called SapFix that its engineers are building. From a report: SapFix, which is still under development, is designed to generate fixes automatically for specific bugs before sending them to human engineers for approval. Facebook, which announced the tool today ahead of its Scale conference in San Jose, California, for developers building large-scale systems and applications, calls SapFix an "AI hybrid tool." It uses artificial intelligence to automate the creation of fixes for bugs that have been identified by its software testing tool Sapienz, which is already being used in production. SapFix will eventually be able to operate independently from Sapienz, but for now it's still a proof-of-concept that relies on the latter tool to pinpoint bugs first of all. SapFix can fix bugs in a number of ways, depending on how complex they are, Facebook engineers Yue Jia, Ke Mao and Mark Harman wrote in a blog post announcing the tools. For simpler bugs, SapFix creates patches that revert the code submission that introduced them. In the case of more complicated bugs, SapFix uses a collection of "templated fixes" that were created by human engineers based on previous bug fixes.

Python creator Guido van Rossum retired in July, but he's been pulled back in to resolve a debate about politically incorrect language. The Register reports: Like other open source communities, Python's minders have been asked whether they really want to continue using the terms "master" and "slave" to describe technical operations and relationships, given that the words remind some people of America's peculiar institution, a historical legacy that fires political passions to this day. Last week Victor Stinner, a Python developer who works for Red Hat, published four pull requests seeking to change "master" and "slave" in Python documentation and code to terms like "parent," "worker," or something similarly anodyne. "For diversity reasons, it would be nice to try to avoid 'master' and 'slave' terminology which can be associated to slavery," he explained in his bug report, noting that there have been complaints but they've been filed privately -- presumably to avoid being dragged into a fractious flame war. And when Python 3.8 is released, there will be fewer instances of these terms.

Microsoft Research has pre-published a new paper to be presented at the IEEE Cybersecurity Development Conference 2018 describing their progress on Checked C, "an extension to C designed to support spatial safety, implemented in Clang and LLVM."

From "Checked C: Making C Safe By Extension": Checked C's design is distinguished by its focus on backward-compatibility, incremental conversion, developer control, and enabling highly performant code... Any part of a program may contain, and benefit from, checked pointers. Such pointers are binary-compatible with legacy, unchecked pointers but have explicitly annotated and enforced bounds. Code units annotated as checked regions provide guaranteed safety: The code within may not use unchecked pointers or unsafe casts that could result in spatial safety violations.

Checked C's bounds-safe interfaces provide checked types to unchecked code, which is useful for retrofitting third party and standard libraries. Together, these features permit incrementally adding safety to a legacy program, rather than making it an all-or-nothing proposition. Our implementation of Checked C as an LLVM extension enjoys good performance, with relatively low run-time and compilation overheads. It is freely available at https://github.com/Microsoft/checkedc and continues to be actively developed.
The extension is enabled as a flag passed to Clang -- the average run-time overhead introduced by adding dynamic checks was 8.6%, though in more than half of the benchmarks the overhead was less than 1%. They also note that from 2012 to 2018, buffer overruns were the leading single cause of CVEs.

Microsoft Research says they're now evaluating Checked C, formalizing a proof of its safety guarantee -- and developing a tool to semi-automatically rewrite legacy C programs.

InfoWorld described the move as a "breakthrough": As expected, Python has climbed into the Top 3 of the Tiobe index of language popularity, achieving that milestone for the first time ever in the September 2018 edition of the index. With a rating of 7.653 percent, Python placed third behind first-place Java, which had a rating of 17.436 percent, and second-place C, rated at 15.447. Python displaced C++, which finished third last month and took fourth place this month, with a rating of 7.394 percent...

Python also has been scoring high in two other language rankings:

- The PyPL Popularity of Programming Language index, where it ranked No. 1 this month, as it has done before, and has had the most growth in the past five years.

- The RedMonk Programming Language Rankings, where Python again placed third.
Tiobe notes that Python's arrival in the top 3 "really took a long time," since it first entered their chart at the beginning of the 1990s. But today, "It is already the first choice at universities (for all kinds of subjects for which programming is demanded) and is now also conquering the industrial world." In February Tiobe also added a new programming language to their index: SQL. (Since "SQL appears to be Turing complete.")

"Other interesting moves this month are: Rust jumps from #36 to #31, Groovy from #44 to #34 and Julia from #50 to #39."

"The JavaScript world could use a bit of classification," reads this year's announcement at StateofJS.com: In 2017 this survey helped us do just that, by collecting data from over 20,000 developers to identify current and upcoming trends. This year, we're asking for your help once more to find out which libraries developers want to learn next, which have the best satisfaction ratings, and much more.
The survey launched in 2016 "mostly to scratch my own itch," its founder explained in a Medium essay. "I wanted to know what libraries were worth learning, and which ones were on the way out." Last year's survey discovered that React was the dominant framework, though the second most-popular framework was "none," with 9,493 JavaScript developers saying they didn't use one. Vue had increased in popularity while Angular lost steam, and developers collectively rating their overall happiness with front-end tools at 3.8 (on a scale up to five).

And more than 28% of the survey's respondent's said they'd used TypeScript, Microsoft's typed superset of JavaScript, and that they'd use it again.

"Feeling like a hack is more common than you might think," writes CNET: In fact, 58 percent of people with technology-focused careers suffer from Impostor Syndrome, according to a new informal study from workplace social media site Blind... Blind's user base includes 44,000 Microsoft employees, 29,000 from Amazon, 11,000 from Google, 8,000 from Uber, 7,000 from Facebook, and 6,000 from Apple, just to name a few. From Aug. 27, 2018 through Sept. 5, 2018, Blind asked its users one question in a survey -- "Do you suffer from Impostor Syndrome?" A total of 10,402 users on Blind responded.

Blind found that 57.55 percent surveyed experienced Impostor Syndrome. Seventy-two percent of Expedia employees say they experienced Impostor Syndrome, the highest among companies with at least 100 employee responses. On the lower end of the spectrum, only 44.45 percent of Apple employees experienced impostor syndrome.

OSNews reports: Terrence Andrew Davis, sole creator and developer of TempleOS (née LoseThos), has passed away at age 48. Davis suffered from mental illness -- schizophrenia -- which had a severe impact on his life. He claimed he created his operating system after having spoken with and receiving instructions from god, and he was a controversial figure, also here on OSNews, for his incomprehensible rants and abrasive style towards OSNews readers and staff. We eventually had to ban him, but our then-editor Kroc Kamen worked with him in 2010 to publish an article about his operating system despite his ban.... I hope he found peace -- wherever he may be.
Davis spent 10 years building "an operating system to talk to God," according to a 2014 profile in Motherboard, which described its welcome screen as "a riot of 16-color, scrolling, blinking text" resembling early DOS-based GUIs. (Wikipedia describes its interface as "a mixture of DOS and Turbo C.") To build his operating system, Terry wrote 121,176 lines of code.

An anonymous reader writes: Davis learned assembly language on a Commodore 64 before he'd graduated from high school. He eventually got a master's degree in electrical engineering from Arizona State University, and as an undergrad he worked briefly at Ticketmaster, programming operating systems. His later life included time in mental hospitals and some homelessness, as well as living at home with his parents after his schizophrenia was diagnosed and treated.

In 2014 Motherboard pieced together his lifestyle from emailed updates Terry sent from his Ubuntu desktop. They concluded he was living on disability, and spent most of his time coding, surfing the web, "or using the output from the National Institute of Standards and Technology randomness beacon to talk to God -- he posts the results on his webpage as 'Terry Davis' Rants.'" Their article describes him as "God's lonely programmer," saying Davis "offered the world a temple to a God who speaks only to him, and is still waiting for everyone else to listen."
Terry's death was confirmed by a local Oregon newspaper, and the official web site for TempleOS now also includes this death notice:

In the wake of Terry A. Davis' passing his family has requested supporters of his donate to "organizations working to ease the pain and suffering caused by mental illness" such as

• The Brain & Behavior Research Foundation
• National Alliance on Mental Illness

An anonymous reader quotes a report from Motherboard: Wednesday, Valve, the company that operates the huge online video game store Steam, shared more details about how it plans to control and moderate the ever-increasing number of games published on its platform. In the post published Wednesday, Valve shared more details about how it determines what it considers "outright trolling." "It is vague and we'll tell you why," Valve wrote. "You're a denizen of the internet so you know that trolls come in all forms. On Steam, some are simply trying to rile people up with something we call 'a game shaped object' (ie: a crudely made piece of software that technically and just barely passes our bar as a functioning video game but isn't what 99.9% of folks would say is "good.")

Valve goes on to explain that some trolls are trying to scam folks out of their Steam inventory items (digital items that can be traded for real money), while others are trying to generate a small amount of money through a variety of schemes that have to do with how developers use keys to unlock Steam games, while others are trying to "incite and sow discord." "Trolls are figuring out new ways to be loathsome as we write this," Valve said. "But the thing these folks have in common is that they aren't actually interested in good faith efforts to make and sell games to you or anyone. When a developer's motives aren't that, they're probably a troll." One interesting observation Valve shares in the blog post is that it rarely bans individual games from Steam, and more often bans developers and/or publishers entirely. [...] Valve said that its review process for determining that something may be a "troll game" is a "deep assessment" that involves investigating who the developer is, what they've done in the past, their behavior on Steam as a developer, as a customer, their banking information, developers they associate with, and more.

An anonymous reader quotes a report from CNBC: As our global economy increasingly comes to run on technology-enabled rails and every company becomes a tech company, demand for high-quality software engineers is at an all-time high. A recent study from Stripe and Harris Poll found that 61 percent of C-suite executives believe access to developer talent is a threat to the success of their business. Perhaps more surprisingly -- as we mark a decade after the financial crisis -- this threat was even ranked above capital constraints. And yet, despite being many corporations' most precious resource, developer talents are all too often squandered. Collectively, companies today lose upward of $300 billion a year paying down "technical debt," as developers pour time into maintaining legacy systems or dealing with the ramifications of bad software. This is especially worrisome, given the outsized impact developers have on companies' chances of success. Software developers don't have a monopoly on good ideas, but their skill set makes them a uniquely deep source of innovation, productivity and new economic connections. When deployed correctly, developers can be economic multipliers -- coefficients that dramatically ratchet up the output of the teams and companies of which they're a part.

Long time reader theodp writes: According to a Microsoft-commissioned survey, 50% of parents in the U.S. with children aged 18 and under believed coding and computer programming to be the most beneficial subject to their child's future employability ("compared to foreign language skills at 28%"). From the Microsoft Education blog post: "When asked about the technology industry's involvement, 75 percent of parents said they believe big tech companies should be involved in helping schools build kids' digital skills. Many companies, including Microsoft and organizations like Code.org, are working to do just that. Programs like TEALS, which is supported by Microsoft Philanthropies, pairs trained Computer Science professionals from across the technology industry with classroom teachers to team-teach the subject." In 2016, Microsoft partnered with Rhode Island Gov. Gina Raimondo to help bring computer science education to every public K-12 school across the state, an initiative that Raimondo is now touting in her 2018 bid for re-election (political ad).