How To Get Streaming Data From Twitter

I occasionally receive requests to share my Twitter analysis tools. After a few recent requests, it finally occurred to me that it would make sense to create a series of articles that describe how to use Python and the Twitter API to perform basic analytical tasks. Teach a man to fish, and all that.

In this blog post, I’ll describe how to obtain streaming data using Python and the Twitter API.

I’m using twarc instead of tweepy to gather data from Twitter streams. I recently switched to using twarc, because has a simpler interface than tweepy, and handles most network errors and Twitter errors automatically.

In this article, I’ll provide two examples. The first one covers the simplest way to get streaming data from Twitter. Let’s start by importing our dependencies.

from twarc import Twarc
import sys

Next, create a twarc session. For this, you’ll need to create a Twitter application in order to obtain the relevant authentication keys and fill in those empty strings. You can find many guides on the Internet for this. Here’s one.

if __name__ == '__main__':
  consumer_key=""
  consumer_secret=""
  access_token=""
  access_token_secret=""

  twarc = Twarc(consumer_key, consumer_secret, access_token, access_token_secret)

For the sake of brevity, let’s assume search terms will be passed as a list on the command-line. We’ll simply accept that list without checking it’s validity. Your own implementation should probably do more.

  target_list = []
  if (len(sys.argv) > 1):
    target_list = sys.argv[1:]

Finally, we’ll check if we have any search targets. If we do, we’ll create a search query. If not, we’ll attach to the sample stream.

  if len(target_list) > 0:
    query = ",".join(target_list)
    print "Search: " + query
    for tweet in twarc.filter(track = query):
      print_tweet(tweet)
  else:
    print "Getting 1% sample."
    for tweet in twarc.sample():
      print_tweet(tweet)

Here’s a function to print the “text” field of each tweet we receive from the stream.

def print_tweet(status):
  if "text" in status:
    print status["text"]

And that’s it. In just over 20 lines of code, you can attach to a Twitter stream, receive Tweets, and process (or in this case, print) them.

In my second example, incoming Tweet objects will be pushed onto a queue in the main thread, while a second processing thread will pull those objects off the queue and process them. The reason we would want to separate gathering and processing into separate threads is to prevent any blocking by the processing step. Although in this example, simply printing a Tweet’s text out is unlikely to block under normal circumstances, once your processing code becomes more complex, blocking is more likely to occur. By offloading processing to a separate thread, your script should be able to handle things such as heavy Tweet volume spikes, writing to disk, communicating over the network, using machine learning models, and working with large frequency distribution maps.

As before, we’ll start by importing dependencies. We’re including threading (for multithreading), Queue (to manage a queue), and time (for time.sleep).

from twarc import Twarc
import Queue
import threading
import sys
import time

The following two functions will run in our processing thread. One will process a Tweet object. In this case, we’ll do exactly the same as in our previous example, and simply print the Tweet’s text out.

# Processing thread
def process_tweet(status):
  if "text" in status:
    print status["text"]

The other function that will run in the context of the processing thread is a function to get items that were pushed into the queue. Here’s what it looks like.

def tweet_processing_thread():
  while True:
    item = tweet_queue.get()
    process_tweet(item)
    tweet_queue.task_done()

There are also two functions in our main thread. This one implements the same logic for attaching to a Twitter stream as in our first example. However, instead of calling process_tweet() directly, it pushes tweets onto the queue.

# Main thread
def get_tweet_stream(target_list, twarc):
  if len(target_list) > 0:
    query = ",".join(target_list)
    print "Search: " + query
    for tweet in twarc.filter(track = query):
      tweet_queue.put(tweet)
  else:
    print "Getting 1% sample."
    for tweet in twarc.sample():
      tweet_queue.put(tweet)

Now for our main function. We’ll start by creating a twarc object, and getting command-line args (as before):

if __name__ == '__main__':
  consumer_key=""
  consumer_secret=""
  access_token=""
  access_token_secret=""

  twarc = Twarc(consumer_key, consumer_secret, access_token, access_token_secret)

  target_list = []
    if (len(sys.argv) > 1):
      target_list = sys.argv[1:]

Next, let’s create the queue and start our processing thread.

  tweet_queue = Queue.Queue()
  thread = threading.Thread(target=tweet_processing_thread)
  thread.daemon = True
  thread.start()

Since listening to a Twitter stream is essentially an endless loop, let’s add the ability to catch ctrl-c and clean up if needed.

  while True:
    try:
      get_tweet_stream(target_list, twarc)
    except KeyboardInterrupt:
      print "Keyboard interrupt..."
      # Handle cleanup (save data, etc)
      sys.exit(0)
    except:
      print("Error. Restarting...")
      time.sleep(5)
      pass

If you want to observe a queue buildup, add a sleep into the process_tweet() function, and attach to a stream with high enough volume (such as passing “trump” as a command-line parameter). Have fun listening to Twitter streams!



Further Analysis Of The Finnish Themed Twitter Botnet

In a blog post I published yesterday, I detailed the methodology I have been using to discover “Finnish themed” Twitter accounts that are most likely being programmatically created. In my previous post, I called them “bots”, but for the sake of clarity, let’s refer to them as “suspicious accounts”.

These suspicious accounts all follow a subset of recommended profiles presented to new Twitter users. In many cases, these automatically created Twitter accounts follow exactly 21 users. The reason I pursued this line of research was because it was similar to a phenomenon I’d seen happening in the US earlier last year. Check this post for more details about that case.

In an attempt to estimate the number of accounts created by the automated process described in my previous post, I ran the same analysis tool against a list of 114 Twitter profiles recommended to new Finnish users. Here is the list.

juhasipila
TuomasEnbuske
alexstubb
hsfi
mikko
rikurantala
yleuutiset
jatkoaika
smliiga
Valavuori
SarasvuoJari
niinisto
iltasanomat
Tami2605
KauppalehtiFi
talouselama
TeemuSel8nne
nokia
HeikelaJussi
hjallisharkimo
Linnanahde
tapio_suominen
vrantanen
meteorologit
tikitalk10
yleurheilu
JaajoLinnonmaa
hirviniemi
pvesterbacka
taloussanomat
TuomasKyr
MTVUutiset
Haavisto
SuomenKuvalehti
MikaelJungner
paavoarhinmaki
KajKunnas
SamiHedberg
VilleNiinisto
HenkkaHypponen
SaskaSaarikoski
jhiitela
Finnair
TarjaHalonen
leijonat
JollaHQ
filsdeproust
makinenantti
lottabacklund
jyrkikasvi
JethroRostedt
Ulkoministerio
valtioneuvosto
Yleisradio
annaperho
liandersson
pekkasauri
neiltyson
villetolvanen
akiriihilahti
TampereenPoika
madventures
Vapaavuori
jkekalainen
AppelsinUlla
pakalupapito
rakelliekki
kyleturris
tanelitikka
SlushHQ
arcticstartup
lindaliukas
goodnewsfinland
docventures
jasondemers5
Retee27
H_Kovalainen
ipaananen
FrenzziiiBull
ylenews
digitoday
jraitamaa
marmai
MikaVayrynen
LKomarov
ovi8
paulavesala
OsmoSoininvaara
juuuso
JaanaPelkonen
saaraaalto
yletiede
TimoHaapala
Huuhkajat
ErvastiPekka
JussiPullinen
rsiilasmaa
moia
Palloliitto
teroterotero
ARaanta31
kirsipiha
JPohjanpalo
startupsauna
aaltoes
Villebla
MariaVeitola
merjaya
MikiKuusi
MTVSportfi
EHaula
svuorikoski
andrewickstroem
kokoomus

For each account, my script saved a list of accounts suspected of being automatically created. After completing the analysis of these 114 accounts, I iterated through all collected lists in order to identify all unique account names across those lists.

Across the 114 recommended Twitter profiles, my analysis identified 5631 unique accounts. Here are the (first twenty) age ranges of the most recently created accounts:

All age ranges

Age ranges of all suspicious Twitter accounts identified by my script

It has been suggested (link in Finnish) that these accounts appeared when a popular game, Growtopia, asked its players to follow their Twitter account after a game outage, and those new accounts started following recommended Twitter profiles (including those of Haavisto and Niinistö). In order to check if this was the case, I collected a list of accounts following @growtopiagame, and checked for accounts that appear on both that list, and the list of suspicious accounts collected in my previous step. That number was 3. This likely indicates that the accounts my analysis identified aren’t players of Growtopia.



Someone Is Building A Finnish-Themed Twitter Botnet

Finland will hold a presidential election on the 28th January 2018. Campaigning just started, and candidates are being regularly interviewed by the press and on the TV. In a recent interview, one of the presidential candidates, Pekka Haavisto, mentioned that both his Twitter account, and the account of the current Finnish president, Sauli Niinistö had recently been followed by a number of bot accounts. I couldn’t resist investigating this myself.

I wrote a tool to analyze a Twitter account’s followers. The Twitter API only gives me access to the last 5000 accounts that have followed a queried account. However, this was enough for me to find some interesting data.

As I previously wrote, newly created bulk bot accounts often look very similar. I implemented some logic in my follower analysis tool that attempts to identify bots by looking for a combination of the following:

  • Is the account still an “egg” (default profile settings, default picture, etc.)?
  • Does the account follow exactly 21 other accounts?
  • Does the account follow very few accounts (less than 22)?
  • Does the account have a bot-like name (a string of random characters)?
  • Does the account have zero followers?
  • Has the account tweeted zero times?

Each of the above conditions give a score. If the total of all scores exceeds an arbitrary value, I record the name of the account.

I ran this tool against @Haavisto and @niinisto Twitter accounts and found the following:
Matches for @Haavisto account: 399
Matches for @niinisto account: 330

In both cases, the accounts in question were by-and-large under 2 months old.

Haavisto bot account age ranges

Account age ranges for bots following @Haavisto

 

Niinisto account bot follower age ranges

Account age ranges for bots following @niinisto

Interestingly, I checked the intersection between these two groups of bots. Only 49 of these accounts followed both @Haavisto and @niinisto.

Checking a handful of the flagged accounts manually using the Twitter web client, I quickly noticed that they all follow a similar selection of high-profile Finnish twitter accounts, including accounts such as:

Tuomas Enbuske (@TuomasEnbuske) – a Finnish celebrity
Riku Rantala (@rikurantala) – host of Madventures
Sauli Niinistö (@niinisto) – Finland’s current president
Juha Sipilä (@juhasipila) – Finland’s prime minister
Alexander Stubb (@alexstubb) – Former prime minister of Finland
Pekka Haavisto (@Haavisto) – presidential candidate
YLE (@yleuutiset) – Finland’s equivalent of the BBC
Kauppalehti (@KauppalehtiFi) – a popular Finnish newspaper
Ilta Sanomat (@iltasanomat) – a popular Finnish newspaper
Talous Sanomat (@taloussanomat) – a prominent financial news source
Helsingin Sanomat (@hsfi) – Helsinki’s local newspaper
Ilmatieteen laitos (@meteorologit) – Finnish weather reporting source

What the bots are following

All the bots were following similar popular Finnish Twitter accounts, such as these.

Running the same analysis tool against Riku Rantala’s account yielded similar results. In fact, Riku has been the recipient of 660 new bot followers (although some of them were added on previous waves, judging by the account ages).

Account age ranges for bots following @rikurantala

I have no doubt that the other accounts listed above (and a few more) have recently been followed by several hundred of these bots.

By the way, running the same analysis against the @realDonaldTrump account only found 220 new bots. To verify, I also ran the tool against @mikko yielding a count of 103 bots, and against @rsiilasmaa I found only 38.

It seems someone is busy building a Finnish-themed Twitter botnet. We don’t yet know what it will be used for.



Some Notes On Meltdown And Spectre

The recently disclosed Meltdown and Spectre vulnerabilities can be viewed as privilege escalation attacks that allow an attacker to read data from memory locations that aren’t meant to be accessible. Neither of these vulnerabilities allow for code execution. However, exploits based on these vulnerabilities could allow an adversary to obtain sensitive information from memory (such as credentials, certificates, credit card information, etc.)

Exploits based on the Meltdown and Spectre vulnerabilities work by exploiting a feature of modern processors known as speculative execution (originally proposed by R. M. Tomasulo in 1967). Explained in simple terms, these exploits perform roughly the following four steps:

  • Flush or evict cache lines
  • Run code that causes the processor to perform speculative operations
  • Measure time to access certain cache locations known to contain secret data
  • Infer what that data was from measured access times

What is speculative execution?

Speculative execution is a technique used by high-speed processors in order to increase performance by guessing likely future execution paths and prematurely executing the instructions in them.

Although the results of these speculative execution paths are discarded if program control flow fails to reach them, they can leave behind observable effects in the system. The Meltdown and Spectre white papers primarily detail techniques whereby processor cache lines are examined in order to infer the results of speculative operations. However, the Spectre paper also suggests several other methods (not necessarily related to examining the cache) to do this, and concludes that “virtually any observable effect of speculatively executed code can be leveraged to leak sensitive information”. The primary techniques used to examine processor cache lines in the Meltdown and Spectre papers are Flush+Reload and Evict+Reload.

How do Flush+Reload and Evict+Reload work?

These techniques work by measuring the time it takes to perform a memory read at an address corresponding to an evicted or flushed cache line. If, after an operation, the monitored cache line was recently accessed, data will exist in the cache and access will be fast. If the cache line was not recently accessed, the read will be slow.

Flush+Reload illustration.

Here’s an example of cache line read times in a Flush+Reload attack. Note the downward spike indicating a faster read at one location. (Source: https://meltdownattack.com/meltdown.pdf)

What is the difference between Flush+Reload and Evict+Reload?

The main difference between these two techniques is the mechanism used for clearing the cache.

Flush+Reload uses a dedicated machine instruction, e.g., x86’s clflush, to evict the cache lines.

Evict+Reload forces contention on the cache set that stores the line, causing the processor to discard the contents of that cache line. Evict+Reload techniques are typically used when access to clflush is unavailable (e.g. from JavaScript).

Example code in the Spectre white paper (written in C) uses _mm_clflush() to perform a Flush+Reload attack (see Appendix A of the paper for details).

An example of JavaScript Evict+Reload can be found here.  Note that this example is not the actual technique described in the Spectre white paper. The researchers note that the accuracy of performance.now() is intentionally degraded to dissuade timing attacks, so they implemented a more high-resolution timer by spawning separate threads that repeatedly decrement a value in a shared memory location.

Although Meltdown and Spectre utilize the same basic premise, there are a few differences in the details of how they work, and what they can be used for.

Spectre

A number of proof of concepts exists for Spectre.

One proof of concept uses the Linux kernel’s eBPF mechanism in order to execute a code construct in the context of the OS kernel, thus leading to similar type of privilege escalation as presented by Meltdown, but without using kernel exceptions.

Another Spectre proof of concept allows the host kernel memory space of a KVM (Linux Kernel Virtual Machine) to be exposed. This attack, however, needs admin access to the KVM guest image.

One of the most interesting applications of Spectre is a JavaScript-based proof of concept that can access memory from within the browser process it was run. This exploit contains code that, when translated by a JIT, evaluates to a specific set of machine instructions. Execution of these JavaScript-based exploits happen within the context of the browser, and cannot read memory from outside of the browser process. However, these examples can readily be turned into weaponized exploits designed to extract secrets from within a browser’s memory space (such as credentials, certificates, etc.)

Spectre proof of concepts have been shown to work on Intel, ARM, and AMD processors.

Meltdown

Meltdown exploits scenarios where some CPUs allow out-of-order execution of user instructions to read kernel memory. Meltdown uses exception handlers to achieve this.

  • Currently, Meltdown proof of concepts have only been successfully tested on Intel CPUs.
  • Meltdown requires the attacker to execute code on the victim’s system itself.
  • Meltdown can be used to defeat kernel address space layout randomization (KASLR).
  • Meltdown can also be used to read memory from adjacent virtualized containers (Docker, LXC, OpenVZ) on the same physical host that share the same kernel.
  • The Meltdown vulnerability is effectively shut down by the KAISER patch (see below).

Mitigations

Most operating systems have already received an update that includes the KAISER patch. This patch implements a stronger isolation between kernel and user space, thus breaking the techniques that allow the Meltdown vulnerability to be exploited to read kernel memory. With KAISER in place, it is still possible to break KASLR using Meltdown exploitation techniques. However, this technique becomes non-trivial.

Since some Spectre exploits are likely to target browsers, we expect browser vendors will patch against these attacks in the near future. These patches will likely disrupt scripts’ ability to accurately record timings, and thus break the Evict+Reload portion of the attack. Depending on when you read this, Firefox may have already been patched against some of these attack vectors.

Update: iOS 11.2.2 also patches Safari against Spectre.

KAISER performance concerns

The KAISER patch is known to affect system performance. Performance impacts will be higher on software that performs a lot of system calls. Actual performance impact numbers will depend on the software and environment in question. It is likely that certain server operating environments will be affected the most. Home machines will likely not see any significant impact.

Will the KAISER patch slow down cryptomining?

Mining, whether CPU- or GPU-based shouldn’t be affected (there shouldn’t be any syscalls in mining loops). Monero (a CPU-based miner) network hashrate appears largely unchanged since the patch.

Detection of Meltdown and Spectre

Kernel memory violations are generated relatively infrequently by regular software. However, any process attempting to exploit Meltdown would generate thousands of such violations over a short duration. Capsule8 suggests that a system designed to monitor for an abundance of segmentation violations for kernel memory addresses (especially from the same PID) could be used to detect meltdown exploits in action.

Endgame recommends monitoring for cache timing attacks using hardware performance counters. In their blog, they examine methods to detect signs of Meltdown exploitation using TSX counters, page flush counters, and by counting last-level-cache (LLC) micro operations. They also examine how it might be possible to detect Spectre attacks by recording speculative branch execution leaks.

Future

It is likely that software exploiting either Meltdown or Spectre will gather secrets as an intermediate step of a longer attack chain (e.g. read credentials from memory and use them to elevate a process). Although patches against Meltdown have already been released for current modern operating systems, there are plenty of legacy systems in the wild, and many users wait a long time, or don’t bother patching at all. Just as old SMB vulnerabilities were leveraged by WannaCry in the not-too-distant past, we’d expect Meltdown to be fair game in the future.

In the near future it is possible that new findings arise around speculative execution implementations, especially on the Intel platform.



Don’t Let An Auto-Elevating Bot Spoil Your Christmas

Ho ho ho! Christmas is coming, and for many people it’s time to do some online shopping.
Authors of banking Trojans are well aware of this yearly phenomenon, so it shouldn’t come as a surprise that some of them have been hard at work preparing some nasty surprises for this shopping season.

And that’s exactly what TrickBot has just gone and done. As one of the most prevalent banking malware for Windows nowadays, we’ve recently seen it diversify into attacking Nordic banks. We’ve blogged about that a couple of times already.

As usual, the Trojan is being delivered via spam campaigns. According to this graph, based on our telemetry, most spam was distributed between Tuesday afternoon and Wednesday morning:

trickbot_spam_graph_20171213

The spam emails we’ve seen typically have a generic subject like “Your Payment – 1234”, a body with nothing but “Your Payment is attached”, and indeed an attachment which is a Microsoft Word document with instructions in somewhat poor English…

trickbot_spam_word_doc

Clicking the button will not reveal any document content, but launch a macro that will eventually download and run the TrickBot payload.
Same old trick, but some people who have just bought a Christmas gift might still fall for it and end up with another ‘gift’ installed on their computer.

And that ‘gift’ is the most interesting part of this story. The newest payload underwent some changes which are, well, remarkable…

Targets

Since its initial appearance during Fall 2016, the actors have been actively developing the malware, and are constantly expanding and changing the targets. Here a short summary of the recently spotted changes:

  • Removed: banks in Australia, New Zealand, Argentina, Italy
  • Changed: a few Spanish, Austrian and Finnish targets are now found in the Dynamic Injection list (adding interception code to the actual web page) instead of using Static Injection (replacing the complete web page)
  • Added: new banks, particularly in France, Belgium and Greece.

Anti-sandbox checks

Up till now, we were not aware of any features in TrickBot that were checking if the malware is run in a virtual machine or a sandboxed environment used for automatic analysis. The new version has introduced a few simple checks against some known sandboxes by calling GetModuleHandle for the following DLLs:

trickbot_antisandbox

(More info about every DLL can be found here)

If any of these modules are found, the payload just quits.

Interestingly, we have also found a few encrypted strings that seem to indicate detection of the Windows virtual machine images that Microsoft provides for web developers to test their code in Internet Explorer and Edge, however, these strings are not used anywhere (yet). Let’s see if the actors will expand their sandbox evasion attempts in a future version.

trickbot_test_vm

Auto-elevation

But we have saved the best for last. When the payload was running, we noticed that it didn’t run with user rights, as it always did before. Instead, it was running under the SYSTEM account, i.e. with full system privileges. There was no UAC prompt during the infection sequence, so TrickBot must have used an auto-elevation mechanism to gain admin rights.

A little search in the disassembly quickly revealed an obvious clue:

trickbot_elevation_1

Combined with a few hard-coded CLSIDs …

trickbot_elevation_2

… we found out that the actors have implemented a UAC bypass which was (as far as we are aware of) publicly disclosed only a few months ago. The original discovery is explained here:
https://msitpros.com/?p=3960
And later implemented as a standalone piece of code, and most likely the main inspiration for the TrickBot coders:
https://gist.github.com/hfiref0x/196af729106b780db1c73428b5a5d68d

In short: this bypass is a re-implementation of a COM interface to launch ShellExec with admin rights, and it is used in a standard Windows component “Component Manager Administrator Kit” to install network connections on machine level.

It works everywhere from Windows 7 up to the latest Windows 10 version 1709 with default UAC settings, and considering it’s basically a Windows feature, probably hard to address. In other words, perfect for usage in malware, and it wouldn’t surprise us if we’ll see the same bypass in more families soon.

Thanks to Päivi for the spam graph.

 



Necurs’ Business Is Booming In A New Partnership With Scarab Ransomware

Necurs’ spam botnet business is doing well as it is seemingly acquiring new customers. The Necurs botnet is the biggest deliverer of spam with 5 to 6 million infected hosts online monthly, and is responsible for the biggest single malware spam campaigns. Its service model provides the whole infection chain: from spam emails with malicious malware downloader attachments, to hosting the payloads on compromised websites.

necurs_other

Necurs is contributing a fair bit to the malicious spam traffic we observe.

The Necurs botnet is most renown for distributing the Dridex banking Trojan, Locky ransomware, and “pump-and-dump” penny-stock spam. Since 2016 it has expanded its deliverables beyond these three and have included other families of ransomware, such as GlobeImposter and Jaff, and the banking trojan Trickbot to its customer base, with Locky being its brand-image malware deliverable with multiple malware spam campaigns per week.

This morning at 9AM (Helsinki time, UTC +2) we observed the start of a campaign with malicious .vbs script downloaders compressed with 7zip. The email subject lines are “Scanned from (Lexmark/HP/Canon/Epson)” and the attachment filename is formatted as “image2017-11-23-(7 random digits).7z“.

The final payload (to our surprise) was Scarab ransomware, which we haven’t seen previously delivered in massive spam campaigns. Scarab ransomware is a relatively new ransomware variant first observed last June, and its code is based on the open source “ransomware proof-of-concept” called HiddenTear.

This version doesn’t change the file names, but appends a new file extension to the encrypted files with “.[suupport@protonmail.com].scarab”, and drops the following ransom note after the encryption:

ransomnote

The spam campaigns from Necurs are following the same format from campaign to campaign, consisting of social engineering subject line themes varying from financial to office utilities, with very minimal text body contents and spiced up usually with malicious attachments, sometimes just URLs. And as the simple social engineering themes are effective, Necurs tends to re-use the spam themes in its campaigns, sometimes within a rather short cycle. In this particular case, the subject lines used in this spam campaign were last seen in a Locky ransomware campaign exactly two weeks ago, the only difference being the extension of the attached downloader.

locky_scarab

This has already given Scarab-ransomware a massive popularity bump, according to ransomware submissions ID ransomware.

We’re interested to see the future affiliations of this massive botnet and observe how it’s able to change the trends and popularity of malware types and certain families. In the meanwhile, we’ll keep blocking these threats, keeping our customers safe.

IOCs:

b4a671ec80135bfb1c77f5ed61b8a3c80b2b6e51
7ac23eee5e15226867f5fbcf89f116bb01933227
d31beec9e2c7b312ecedb594f45a9f5174155c68
85dc3a0b833efb1da2efdcd62fab565c44f22718
da1e2542b418c85f4b57164e46e04e344db58ab8
a6f1f2dd63d3247adb66bd1ff479086207bd4d2b
14680c48eec4e1f161db1a4a990bd6833575fc8e
af5a64a9a01a9bd6577e8686f79dce45f492152e
c527bc757a64e64c89aaf0d9d02b6e97d9e7bb3d
3f51fb51cb1b9907a7438e2cef2e538acda6b9e9
b0af9ed37972aab714a28bc03fa86f4f90858ef5
6fe57cf326fc2434c93ccc0106b7b64ec0300dd7
http://xploramail.com/JHgd476?
http://miamirecyclecenters.com/JHgd476?
http://hard-grooves.com/JHgd476?
http://xploramail.com/JHgd476?
http://atlantarecyclingcenters.com/JHgd476?
http://pamplonarecados.com/JHgd476?
http://hellonwheelsthemovie.com/JHgd476?


RickRolled by none other than IoTReaper

IoT_Reaper overview

IoT_Reaper, or the Reaper in short, is a Linux bot targeting embedded devices like webcams and home router boxes. Reaper is somewhat loosely based on the Mirai source code, but instead of using a set of admin credentials, the Reaper tries to exploit device HTTP control interfaces.

It uses a range of vulnerabilities (a total of ten as of this writing), from years 2013-2017. All of the vulnerabilities have been fixed by the vendors, but how well are the actual devices updated is another matter. According to some reports, we are talking about a ballpark of millions of infected devices.

In this blogpost, we just wanted to add up some minor details to good reports already published by Netlab 360 [1], CheckPoint [2], Radware [3] and others.

Execution overview

When the Reaper enters device, it does some pretty intense actions in order to disrupt the devices monitoring capabilities. For example, it just brutally deletes a folder “/var/log” with “rm -rf”.
Another action is to disable the Linux watchdog daemon, if present, by sending a specific IOCTL to watchdog device:

watchdog

After the initialization, the Reaper spawns a set of processes for different roles:

  • Poll the command and control servers for instructions
  • Start a simple status reporting server listening on port 23 (telnet)
  • Start apparently unused service in port 48099
  • Start scanning for vulnerable devices

All the child processes run with a random name, such as “6rtr2aur1qtrb”.

String obfuscation

The Reaper’s spawned child processes use a trivial form of string obfuscation, which is surprisingly effective. The main process doesn’t use any obfuscation, but all child processes use this simple scheme when they start executing. Basically, it’s a single-byte XOR (0x22), but the way the data is arranged in memory makes it a bit challenging to connect the data to code.

Main process allocates a table in heap and copies the XOR-encoded data in. Later when the child processes want to reference to particular encoded data, it decodes it in heap and references the decoded data with a numeric index. After usage, the data is decoded back to its original form.

The following screenshot is a good presentation of the procedure:

decrypt

Command and Control

The Reaper polls periodically a fixed set of C2 servers:

weruuoqweiur.com, e.hl852.com, e.ha859.com and 27.102.101.121

The control messages and replies are transmitted over a clear-text HTTP, and the beacons are using the following format:

  /rx/hx.php?mac=%s%s&type=%s&port=%s&ver=%s&act=%d

The protocol is very simple: basically there are only two major functions – shutdown or execute arbitrary payload using the system shell.

Port scanning

One of the child processes starts to scan for vulnerable victims. In addition to randomly generated IP addresses, Reaper uses nine hard-coded addresses for some unkown reason. The addess is scanned with a set of apparently random-looking set of ports, and then with a set of bit more familiar ports:

80, 81, 82, 83, 84, 88, 1080, 3000, 3749, 8001, 8060, 8080, 8081, 8090, 8443, 8880, 10000

In fact, the randomish ports are just byte-swapped presentation of the above port list. So for example, 8880 = 0x22b0 turns to 0xb022 = 45090. The reason for this is still unknown.

It is possible that the author was just lazy and left off some endianness handling code, or maybe it is some other error in the programming logic. Some of the IoT-devices are big-endian, so the ports need to be swapped in order to use them with socket code.

Screenshot of the hard-coded list of ports:

ports

This is the list of hard-coded IP-addresses:

217.155.58.226
85.229.43.75
213.185.228.42
218.186.0.186
103.56.233.78
103.245.77.113
116.58.254.40
201.242.171.137
36.85.177.3

Exploitation

If the Reaper finds promising victim, it next tries to send HTTP-based exploit payload to the target. A total of ten different exploits have been observed so far, and they are related to IoT devices HTTP-based control interface. Here’s a list of the targeted vulnerabilities and HTTP requests associated with them:

1 – Unauthenticated Remote Command Execution for D-Link DIR-600 and DIR-300

Exploit URI: POST /command.php HTTP/1.1

 

2 – CVE-2017-8225: exploitation of custom GoAhead HTTP server in several IP cameras

GET /system.ini?loginuse&loginpas HTTP/1.1

 

3 – Exploiting Netgear ReadyNAS Surveillance unauthenticated Remote Command Execution vulnerability

GET /upgrade_handle.php?cmd=writeuploaddir&uploaddir=%%27echo+nuuo+123456;%%27 HTTP/1.1

 

4 – Exploiting of Vacron NVR through Remote Command Execution

GET /board.cgi?cmd=cat%%20/etc/passwd HTTP/1.1

 

5 – Exploiting an unauthenticated RCE to list user accounts and their clear text passwords on D-Link 850L wireless routers

POST /hedwig.cgi HTTP/1.1

 

6 – Exploiting a Linksys E1500/E2500 vulnerability caused by missing input validation

POST /apply.cgi HTTP/1.1

 

7 – Exploiting of Netgear DGN DSL modems and routers using an unauthenticated Remote Command Execution

GET /setup.cgi?next_file=netgear.cfg&todo=syscmd&curpath=/&currentsetting.htm=1cmd=echo+dgn+123456 HTTP/1.1

 

8 – Exploiting of AVTech IP cameras, DVRs and NVRs through an unauthenticated information leak and authentication bypass

GET /cgi-bin/user/Config.cgi?.cab&action=get&category=Account.* HTTP/1.1

 

9 – Exploiting DVRs running a custom web server with the distinctive HTTP Server header ‘JAWS/1.0’.

GET /shell?echo+jaws+123456;cat+/proc/cpuinfo HTTP/1.1

 

10 – Unauthenticated remote access to D-Link DIR-645 devices

POST /getcfg.php HTTP/1.1

 

Other details and The Roll

  • Reaper makes connection checks to google DNS server 8.8.8.8. It won’t run without this connectivity.
  • There is no hard-coded payload functionality in this variant. The bot is supposedly receiving the actual functionality, like DDoS instructions, over the control channel.
  • The code contains an unused rickrolling link (yes, I was rickrolled)

Output from IDAPython tool that dumps encoded strings (rickrolling is the second one):

rickroll

Sample hash

Analysis on this post is based on a single version of the Reaper (md5:37798a42df6335cb632f9d8c8430daec)

References

[1] http://blog.netlab.360.com/iot_reaper-a-rappid-spreading-new-iot-botnet-en/
[2] https://research.checkpoint.com/new-iot-botnet-storm-coming/
[3] https://blog.radware.com/security/2017/10/iot_reaper-botnet/



Facebook Phishing Targeted iOS and Android Users from Germany, Sweden and Finland

Two weeks ago, a co-worker received a message in Facebook Messenger from his friend. Based on the message, it seemed that the sender was telling the recipient that he was part of a video in order to lure him into clicking it.

Facebook Messenger message and the corresponding Facebook Page

The shortened link was initially redirecting to Youtube.com, but was later on changed to redirect to yet another shortened link – po.st:

Changes in the Picsee short link

The po.st shortened link supported two types of redirection links – original link and smart links. If the device that accessed the URL was running in iOS or Android, it was redirected to the utm.io shortened link, otherwise it was redirected to smarturl.it.

The short link with the smart links

So for the iOS and Android users, they were served with the following phishing page:

Phishing page for utm.io short link

For the rest of the devices, the users ended up with the smarturl.it link that went through several redirections which eventually led to contenidoviral.net. That page contained an ad-affiliate URL which redirected to mobusi.com, a mobile advertising company.

Phishing page’s ad-affiliate URL

Based on the data from the links, the campaign began last October 15th when it targeted mostly Swedish users. On the 17th, it moved to targeting Finnish users. Then from 19th onwards, it mostly went after German users.

The total number of clicks for the entire campaign reached almost 200,000, where close to 80% of the visitors were from Germany, Sweden and Finland.

Statistics from po.st tracking page

The campaign ran for two weeks with a main motive of stealing Facebook credentials from iOS and Android users. The cybercriminals used those stolen credentials to spread the malicious links, and subsequently gather more credentials. However, while in the process of stealing the credentials, the cybercriminals also attempted to earn from other non-iOS and non-Android users through ad-fraud.

This practice of using email addresses in place of unique names as account credentials creates a big opportunity for phishers. Just by launching this Facebook phishing campaign, they can mass harvest email and password credentials that are later on used for secondary attacks such as gaining access to other systems or services that could have a bigger monetary value because of password reuse.

We highly recommend the affected users to change their passwords as soon as possible, including other systems and services where the same compromised password was used.

URLs:

  • hxxp://lnk[.]pics/19S3Y
  • hxxp://lnk[.]pics/18JDK
  • hxxp://lnk[.]pics/196OV
  • hxxp://lnk[.]pics/18XH7
  • hxxp://lnk[.]pics/196PN
  • hxxp://lnk[.]pics/19LBP
  • hxxp://lnk[.]pics/18YZV
  • hxxp://lnk[.]pics/18QZW
  • hxxp://lnk[.]pics/196PA
  • hxxp://lnk[.]pics/19XK7
  • hxxp://lnk[.]pics/18HFX
  • hxxp://lnk[.]pics/19S3L
  • hxxp://lnk[.]pics/18J7S
  • hxxp://lnk[.]pics/19XKF
  • hxxp://lnk[.]pics/19K94
  • hxxp://lnk[.]pics/19LBW
  • hxxp://pics[.]ee/188g7
  • hxxp://pics[.]ee/18cdl
  • hxxp://po[.]st/ORyChA
  • hxxp://smarturl[.]it/02xuof
  • hxxp://utm[.]io/290459
  • hxxp://at.contenidoviral[.]net


The big difference with Bad Rabbit

Bad Rabbit is the new bunny on the ransomware scene. While the security community has concentrated mainly on the similarities between Bad Rabbit and EternalPetya, there’s one notable difference which has not yet gotten too much attention. The difference is that Bad Rabbit’s disk encryption works.

EternalPetya re-used the custom disk encryption method from the original Petya. Although it didn’t implement the actual ECDH key delivery mechanism, it installed the Petya boot loader, and effectively just rendered the machine useless.

Petya’s disk encryption had one specific weakness: it only encrypted some parts of the key file system structures, not the whole disk. This design obviously lead to speculations about whether it is possible to recover the disk using a known clear-text attack, and in fact researchers have made significant progress in investigating this recovery technique.

At least on the surface, things look quite different with Bad Rabbit. Instead of using a custom encryption mechanism, it follows the current trend in the ransomware community of leveraging known legitimate encryption tools.

Bad Rabbit uses DiskCryptor, a full disk partition encryption software for locking the user disk. The ransomware ships with an unmodified DiskCryptor driver (borrowed from ReactOS) and implements relevant parts of the DiskCryptor user-mode code for communicating with the driver. The ReactOS driver is a signed, valid driver, so it gets loaded by the Windows cleanly with the elevated privileges required by Bad Rabbit’s fake Flash installer dropper.

Key generation

Bad Rabbit uses Windows’ crypto API to generate random key data for the disk encryption. This key data is converted to a human-readable encryption key, which is a 32-bytes long ASCII encoded string, presenting around 165 bits of entropy for the stream cipher AES used by DiskCryptor.

The following screenshot represents the random disk encryption key as it is being generated by the malware:

asquiring_password

Running dispci.exe under user-mode debugger

Along with the random key, Bad Rabbit packages some other information like the victim computer name and domain, then forms the so-called installation key that will be presented after the reboot. In Petya, this code was relatively short code that was protected by the public EC key. In Bad Rabbit, it is a much longer blob of data that is protected by the RSA public key shipped with the malware installer.

After packaging the installation key, the random key data is just discarded. The installation key is written to the disk so that the Bad Rabbit boot loader can present it on the boot screen:

decrypting_disk

Bad Rabbit’s boot loader

The user is expected to grab the installation key from the boot screen, paste it to the attacker’s TOR site, which then will use its own private RSA key to extract the 32-bytes password (typed in the screenshot above).

After the boot screen

Because the disk encryption software used is pretty much similar to any other disk encryption used by businesses around the globe, like TrueCrypt or Microsoft BitLocker, the disk is in fact *still* encrypted, although it has been mounted by the DiskCryptor driver transparently.

So if the user now reboots the machine, the same password prompt will be presented as long as the disk decryption routine is initiated. The DECRYPT tool referenced by the boot loader is actually just a shortcut to the dispci.exe tool dropped by the malware. This tool borrows code from legit DiskCryptor sources for implementing the relevant parts of the code for communicating with the disk encryption driver.

Even though all this is quite apparent by just looking at the code, we wanted to demonstrate the encryption scheme by catching the password inline, at the time it was generated by Bad Rabbit. When this password is used for unlocking the machine, it is possible to install the real DiskCryptor GUI tool and initiate the disk decryption process.

correct_pw

Using DiskCryptor to verify encrypted volume

DiskCryptor identifies the disk presented by the virtual machine (QEMU HARDDISK in the above screenshot) as an AES-encrypted volume, and accepts the random password.

It works now

We have speculated before that the flawed disk encryption in EternalPetya was due to problems in the malware development process. Or it may be that they just didn’t care. People will pay anyways, right?

Whatever was the reason, they have now fixed this issue (if they are the same group of malware developers, which seems to be the consensus in the research community).

At least the developers of Bad Rabbit have noted the recent developments in research on Petya’s disk encryption weaknesses and decided to use something different.

Recovery considerations

As we have demonstrated in this blog post, Bad Rabbit seems to use a sound principle in its disk encryption, a full disk encryption scheme familiar to all businesses.

We don’t yet have the full details of this scheme, so there might be bugs in the implementation. But at least its design enables a strong mechanism for locking the machine until the correct password is really typed to the boot screen.

Disclaimer

For screenshots used in this blog post, we DID NOT go to the attacker TOR site and pay for the recovery key.

The procedures presented in this text DO NOT mean there’s an easy way to unlock the disk protected by the Bad Rabbit. We just present them as proof of the encryption scheme. Catching the password inline to the encryption process is not practical in a general sense, because it requires software that is aware of the exact password generation mechanism prior to the infection. It is used here just for a relatively easily reproducible proof-of-concept.



Following The Bad Rabbit

On October 24th, media outlets reported on an outbreak of ransomware affecting various organizations in Eastern Europe, mainly in Russia and Ukraine. Identified as “Bad Rabbit”, initial reports about the ransomware drew comparisons with the WannaCry and NotPetya (EternalPetya) attacks from earlier this year. Though F-Secure hasn’t yet received any reports of infections from our own customers, we’re actively investigating. And while the investigation is still ongoing, initial results from our analysis did find similarities between Bad Rabbit and the NotPetya ransomware that hit companies late last June.

We think there’s good evidence that suggests the same person or group is responsible for both last June’s NotPetya attacks and what we’re seeing now with Bad Rabbit. Malware authors often learn from what works, so finding the same characteristics in different families is not uncommon. But the similarities we’re seeing here are too much to be just one attacker copying another.

Without getting too technical, here’s a handful of the similarities between NotPetya and Bad Rabbit:

  • Overall code structure is similar
  • File encryption code is VERY similar
  • Similar method of checking existing processes and encrypting files
  • Similar method used to reboot computers
  • Same trick used to launch the malware’s main component as a DLL
  • Identical code used to parse the command line
  • Similar propagation methods, including an identical “library” of other computers found in the network, and use of Mimikatz to gather credentials
  • Out of 113 file extensions used by BadRabbit, 65 are shared with NotPetya (Bad Rabbit has an additional 48)

There are also some notable differences between the two, including:

  • Bad Rabbit doesn’t use EternalBlue/EternalRomance exploit
  • Bad Rabbit doesn’t use PsExec to spread
  • Bad Rabbit also encrypts “home user” files, such as .jpgs
  • Bad Rabbit adds “.encrypted” to the contents of affected files (NotPetya didn’t do this, making it harder to distinguish between encrypted and non-encrypted files)
  • Bad Rabbit’s infection vector is via compromised websites. While NotPetya was reported to be via MeDoc
  • Bad Rabbit brute-forces using a set of predefined credentials to available SMB shares
  • The list of process hashes to be compared to are different from NotPetya. NotPetya compares against Symantec and Kaspersky processes, while Bad Rabbit compares against McAfee and DrWeb

Like NotPetya, Bad Rabbit will display the two ransom note – one for MBR encryption.

Bad Rabbit Message

And a text note for file encryption.

Oops! Your files have been encrypted.

If you see this text, your files are no longer accessible.
You might have been looking for a way to recover your files.
Don't waste your time. No one will be able to recover them without our
decryption service.

We guarantee that you can recover all your files safely. All you
need to do is submit the payment and get the decryption password.

Visit our web service at caforssztxqzf2nm.onion

Your personal installation key#2: [REDACTED]

Users are directed to pay the ransom at a specified payment site, which also provides the amount of the ransom to be paid.

Bad Rabbit Payment Site

A threat description of the Bad Rabbit ransomware is available at Trojan:W32/Rabbad and will be updated as and when more details are confirmed.

In the meantime… our endpoint protection products have a variety of measures baked in that prevent Bad Rabbit infections.


Edited to update: Struckthrough EternalRomance mention above. We have verified the same observations as Cisco Talos Security about EternalRomance exploited by Bad Rabbit.



Twitter Forensics From The 2017 German Election

Over the past month, I’ve pointed Twitter analytics scripts at a set of search terms relevant to the German elections in order to study trends and look for interference. Germans aren’t all that into Twitter. During European waking hours Tweets in German make up less than 0.5% of all Tweets published. Over the last month, […]

2017-09-25

TrickBot In The Nordics, Episode II

The banking trojan TrickBot is not retired yet. Not in the least. In a seemingly never ending series of spam campaigns – not via the Necurs botnet this time – we’ve spotted mails written in Norwegian that appear to be sent by DNB, Norway’s largest bank. The mail wants the recipient to believe that they […]

2017-09-14

Working Around Twitter API Restrictions To Identify Bots

Twitter is by far the easiest social media platform to work with programmatically. The Twitter API provides developers with a clean and simple interface to query Twitter’s objects (Tweets, users, timelines, etc.) and bindings to this API exist for many languages. As an example, I’ve been using Tweepy to write Python scripts that work with Twitter data. […]

2017-08-31

Trump Hating South Americans Hacked HBO

Last week – I read the message “Mr. Smith” reportedly sent to HBO… and it brought up a few questions. And also, it offered some “answers” to questions that I’m often asked. Questions such as “how much money do cyber criminals make?” Here’s the start of the message. First, let’s examine Mr. Smith and his […]

2017-08-24

Break your own product, and break it hard

Hello readers, I am Andrea Barisani, founder of Inverse Path, which is now part of F-Secure. I lead the Hardware Security consulting team within F-Secure’s Cyber Security Services. You may have heard of our USB armory product, an innovative compact computer for security applications that is 100% open hardware, open source and Made in Italy. […]

2017-07-19

Retefe Banking Trojan Targets Both Windows And Mac Users

Based on our telemetry, customers (mainly in the region of Switzerland and Germany) are being targeted by a Retefe banking trojan campaign which uses both Windows and macOS-based attachments. Its massive spam run started earlier this week and peaked yesterday afternoon (Helsinki time). TrendMicro did a nice writeup on this threat earlier this week. The […]

2017-07-14

How EternalPetya Encrypts Files In User Mode

On Thursday of last week (June 29th 2017), just after writing about EternalPetya, we discovered that the user-mode file encryption-decryption mechanism would be functional, provided a victim could obtain the correct key from the malware’s author. Here’s a description of how that mechanism works. EternalPetya malware uses the standard Win32 crypto API to encrypt data. […]

2017-07-04

What Good Is A Not For Profit (Eternal) Petya?

Following up on our post from yesterday, as an intellectual thought experiment, let’s take the position that there’s something to the idea of (Eternal) Petya not being motivated by money/profit. Let’s also just go ahead and imagine that it’s been developed by a nation state. In my mind, it raises the following question: WTF WHY? […]

2017-06-30

(Eternal) Petya From A Developer’s Perspective

In our previous post about Petya, we speculated that the short-cuts, design flaws, and non-functional mechanisms observed in the  malware could have arisen due to it being developed under a tight deadline. I’d now like to elaborate a little on what we meant by that. As a recap, this is what the latest version of Petya […]

2017-06-30

Petya: “I Want To Believe”

There’s been a lot of speculation and conjecture around this “Petya” outbreak. A great deal of it seems to have been fueled by confirmation bias (to us, at least). Many things about this malware don’t add up (at first glance). But it wouldn’t be the first time that’s happened. And yet everyone seems to have […]

2017-06-29