Archive

Posts Tagged ‘Linux’

Extending Microsoft OMS to monitor Squid Proxy running in Linux with a plugin – part 1/3 #MSOMS

November 24, 2016 1 comment

Since Microsoft released OMS, I have been an early adopter and evangelist for the solution. Not only it is simple to deploy but it gives you a full spectrum of many of the workloads you have either on-premises or in the cloud and it does not matter which cloud. Be it Azure, AWS, Google and many others.

So, as I was advising on OMS for a customer, I found that they were running Squid Proxy servers. The Squid proxy server is one of the most famous proxy servers in the world and it has been utilised for years in many organisations. For that reason I then I decided to look at how OMS could leverage the monitoring for Squid.

squi3

As you can see here: https://github.com/Microsoft/OMS-Agent-for-Linux/tree/master/installer/conf/omsagent.d there are already many plugins for OMS to  monitor Windows and many Linux OS as well, DNS, Network, SQL, MySQL, Postgree, VMware, MongoDB, Security, Audit, Change Tracking and so on.

But, there was no Squid plugin and that’s where I brought back my past years of experience as a developer and although that was a long, long time go, I was able to developer in ruby a Squid plugin for Microsoft OMS.

How I developed it?

PART 1 : LOG Files

  1. I started but investigating the squid log on /var/log/squid/access.log and then I research REGEX expressions to extract information out of it. Below is a extract of it

1479696836.902    134 10.1.1.4 TCP_MISS/301 488 open http://cnn.com/ – HIER_DIRECT/151.101.0.73 –
1479696848.110    242 10.1.1.4 TCP_MISS/400 486 open http://www.sydney.com/ – HIER_DIRECT/54.253.253.77 text/html
1479696860.004    407 10.1.1.4 TCP_MISS/301 636 open http://www.7news.com.au/ – HIER_DIRECT/203.84.217.229 text/html

The initial difficult part for me was of to decouple the date/time to get it on a human readable format. So, after long hours of research and playing along, I decided for the following REGEX :

 REGEX =/(?<eventtime>(\d+))\.\d+\s+(?<duration>(\d+))\s+(?<sourceip>(\d+\.\d+\.\d+\.\d+))\s+(?<cache>(\w+))\/(?<status>(\d+))\s+(?<bytes>(\d+)\s+)(?<response>(\w+)\s+)(?<url>([^\s]+))\s+(?<user>(\w+|\-))\s+(?<method>(\S+.\S+))/
(If you have a better one, please feel free to shot me)

 

  1. I then wrote a squidparserlog.rb in ruby to parse the Squid access.log file and turn it into a OMS format
class SquidLogParserLib
require ‘date’
require ‘etc’
require_relative ‘oms_common’
require ‘fluent/parser’
    def initialize(error_handler)
@error_handler = error_handler
end
    REGEX =/(?<eventtime>(\d+))\.\d+\s+(?<duration>(\d+))\s+(?<sourceip>(\d+\.\d+\.\d+\.\d+))\s+(?<cache>(\w+))\/(?<status>(\d+))\s+(?<bytes>(\d+)\s+)(?<response>(\w+)\s+)(?<url>([^\s]+))\s+(?<user>(\w+|\-))\s+(?<method>(\S+.\S+))/
    def parse(line)
      data = {}
time = Time.now.to_f
      begin
REGEX.match(line) { |match|
data[‘Host’] = OMS::Common.get_hostname
          timestamp = Time.at( match[‘eventtime’].to_i() )
data[‘EventTime’] = OMS::Common.format_time(timestamp)
data[‘EventDate’] = timestamp.strftime( ‘%Y-%m-%d’ )
data[‘Duration’] = match[‘duration’].to_i()
data[‘SourceIP’] = match[‘sourceip’]
data[‘cache’] = match[‘cache’]
data[‘status’] = match[‘status’]
data[‘bytes’] = match[‘bytes’].to_i()
data[‘httpresponse’] = match[‘response’]
data[‘bytes’] = match[‘bytes’].to_i()
data[‘url’] = match[‘url’]
data[‘user’] = match[‘user’]
data[‘method’] = match[‘method’]}
rescue => e
@error_handler.logerror(“Unable to parse the line #{e}”)
end
      return time, data
end   #def
   end   #class
3. Finally, I wrote the squid.conf for OMS
# enhanced parse log with date format , which pass the path for the log to the SquidLogParser and tag it as oms.api.Squid. By doing this, you will end up with 11 custom fields in OMS for the LOG TYPE Squid_CL
<source>
type tail
format SquidLogParser
path /var/log/squid/access.log
pos_file /var/opt/microsoft/omsagent/state/var_log_squid_access.pos
tag oms.api.Squid
log_level error
</source>
squid-fields

 

On my next article I will go through the next part, which is getting Squid Proxy Statistics in OMS, along with the full code.

squid2.png

 

Linux Bash Command Line natively on Windows 10

 

If you a Linux fan or like to play around with Linux or are experienced Linux, Mac OS X or Unix developer, this feature is perfect for you: the new Bash feature will give you a Linux root shell, which means that you won’t even need to use sudo to become “Administrator”. As root user you will have full system access, like you have in Linux and Unix systems.

 

6

 

Before we start, is important to understand that an application that you install in the Bash shell is restricted to Bash shell. You won’t be able to access the application from PowerShell, Cmd or Explorer in Windows.

 

The solution

 

To offer a full Bash shell based on Ubuntu, Microsoft worked with Canonical to provide a shell that runs atop the subsystem allowing you to run the Bash shell and the exact same binaries you’d normally run on Ubuntu Linux.

There are some limitations as you won’t be able to install Linux server applications or to start Linux graphical software. The target for the feature are developers who want to run Linux command-line utilities on Windows.

 

Requirements

 

  • Windows 10 Insider Preview Build (minimum build: 14316)
  • Developer Mode activated

 

Getting there in simple 10 steps

 

1. Click on Windows Start, click on Settings and then click on Update & Security

2. On the left menu, click on Windows Update, then click on Advance Options

3. On the Advanced Options window, if you are not already an Windows Insider, click on Get Started and follow the instructions to become an Windows Insider. You will be requested to restart your computer and after that, return to this same windows and select the Fast mode to have at least the 14316 build installed.

0

4. Once the computer have the required minimum build, go back to Update & Security and on the left menu select For Developers.

1

  1. On Windows Start, type Program and Features

2

 

6. On the left panel, click on Turn Windows Feature on or off

3

 

7. Select Windows Subsystem for Linux (Beta) from the list and click on OK.  You’ll be requested to restart your computer,  for the feature to get installed.

4

8. When you computer gets back, click on the Windows Start button and type bash then select the bash command or press enter.

5v2

9. You will be requested to accept the terms of license and after accepting it a bash window will open. Press Y to accept the download of the Bash Ubuntu on Windows application from the Windows Store.

6.png

10. That’s it. You now have a full command-line bash shell based on Linux Ubuntu. You now have access to all the Linux command line software.

Note: As we installed the bash for Ubuntu and because they’re the same binaries as you would have on a normal Linux Ubuntu installation, you can use the same apt-get command you used to run on Ubuntu, to install software from Ubuntu’s repositories.

Notes:

  • In Linux the Bash shell is case-sensitive and the
  • Windows file system is located at /mnt/c in the Bash shell environment.

 

If you find that Microsoft is helping you and your business  as I as do, please help me out by recommending it on Recomazing a new tech platform where socially connected networks store and share trusted recommendations. Please click  here to help our community.

Have Fun!!!

 

 


Categories: Cloud, Microsoft Tags: , , , , , ,

Hyper-V 2016 Tp5: Hyper-V Manager Console new features

Here are some important information and improvements Microsoft released for Hyper-V Manager in Tp5:

  • Alternate credentials support. You can now use a different set of credentials in Hyper-V Manager when you connect to another Windows Server 2016 TP5 or Windows 10 remote host. You can also save these credentials to make it easier to log on again.
  • Previous version management: the New Hyper-V Manager will allow you to manage versions manage computers running Hyper-V on Windows Server 2012/R2 and Windows 8.x client version.
  • Updated management protocolHyper-V Manager has been updated to communicate with remote Hyper-V hosts using the WS-MAN protocol, which permits CredSSP, Kerberos or NTLM authentication. When you use CredSSP to connect to a remote Hyper-V host, you can do a live migration without enabling constrained delegation in Active Directory. The WS-MAN-based infrastructure also makes it easier to enable a host for remote management. WS-MAN connects over port 80, which is open by default.
  • No more Integration services for Windows Virtual Machines. Updates to integration services for Windows guests are distributed through Windows Update. For service providers and hosting companies, this puts the control of applying updates into the hands of the tenants who own the virtual machines. Customers (tenants Administrators) can now update their Windows virtual machines with all updates, including the integration services, using a single method.
  • Integration services for Linux and FreeBSD Virtual Machines. Hyper-V supports both emulated and Hyper-V-specific devices for Linux and FreeBSD virtual machines. Linux Integration Services (LIS) or FreeBSD Integration Services (BIS) , the collection of drivers that are required to run Hyper-V-specific devices, has been added to the Linux kernel and is updated for new releases, but Linux distributions based on older kernels may not have the latest enhancements or fixes. Microsoft provides a download containing installable LIS drivers for some Linux installations based on these older kernels.
    Note: As some Linux distributions include versions of LIS, make sure you install the latest downloadable version of LIS, if applicable, for your installation.

Download:

Hybrid Cloud Computing with Microsoft and Red Hat

November 5, 2015 1 comment

Microsoft and Red Hat announced a partnership that will help customers embrace hybrid cloud computing by providing greater choice and flexibility deploying Red Hat solutions on Microsoft Azure.

redhat-msRed Hat Enterprise Linux will be the preferred option for enterprise Linux workloads on Microsoft Azure.

Microsoft Azure will become a Red Hat Certified Cloud and Service Provider, enabling customers to run their Red Hat Enterprise Linux applications and workloads on Microsoft Azure. Red Hat Cloud Access subscribers will be able to bring their own virtual machine images to run in Microsoft Azure.

Customers will be offered cross-platform, cross-company support spanning the Microsoft and Red Hat offerings in an integrated way, unlike any previous partnership in the public cloud. By co-locating support teams on the same premises, the experience will be simple and seamless, at cloud speed.

Red Hat CloudForms will interoperate with Microsoft Azure and Microsoft System Center Virtual Machine Manager, offering Red Hat CloudForms customers the ability to manage Red Hat Enterprise Linux on both Hyper-V and Microsoft Azure. Support for managing Azure workloads from Red Hat CloudForms is expected to be added in the next few months, extending the existing System Center capabilities for managing Red Hat Enterprise Linux

In addition, Expanding on the preview of .NET on Linux announced by Microsoft in April, developers will have access to .NET technologies across Red Hat offerings, including Red Hat OpenShift and Red Hat Enterprise Linux, jointly backed by Microsoft and Red Hat. Red Hat Enterprise Linux will be the primary development and reference operating system for .NET Core on Linux.

Source: http://www.redhat.com/en/about/press-releases/microsoft-and-red-hat-deliver-new-standard-enterprise-cloud-experiences

Hyper-V Containers bringing speed and scale to the next level in today’s cloud-first world

Last October, Microsoft and Docker, Inc. jointly announced plans to bring containers to developers across the Docker and Windows ecosystems via Windows Server.

Hyper-V Containers will ensure code running in one container remains isolated and cannot impact the host operating system or other containers running on the same host. Applications developed for Windows Server Containers can be deployed as a Hyper-V Container without modification, providing greater flexibility for operators who need to choose degrees of density, agility, and isolation in a multi-platform, multi-application environment.

Docker

The new Microsoft Container technology offers flexibility and choice through Windows Server containers, Linux containers, and Hyper-V containers both in the cloud and on-premises.

Source:http://azure.microsoft.com/blog/2015/04/08/microsoft-unveils-new-container-technologies-for-the-next-generation-cloud

Categories: Cloud Tags: , , , , ,

What’s New in Hyper-V vNext ? Check out at the Infrastructure Saturday event in Brisbane

November 19, 2014 1 comment

Saturday 22nd November

Infrastructure Saturday is a day long event for south east Queensland based IT Professional that work with Microsoft products. This event is an educational, informative & lively day filled with sessions about Microsoft technologies.

Location: Microsoft office, Brisbane, QLD. http://www.infrastructuresaturday.com/

Topics covered in my Session: What’s New in Hyper-V vNext?

 

  • New Virtual Machine Upgrade Process
  • New Integration Components installation method
  • Secure Boot for Linux
  • Distributed Storage Quality of Service (QoS)
  • Hyper-V Backup
  • Hyper-V Virtual Machine Configuration
  • Cluster OS Rolling Upgrade

1

 

 

Linux Support on System Center 2012 SP1

February 19, 2013 7 comments

If you are looking into running Linux on Hyper-V, here are the current Linux versions supported

 

Linux UNIX
Red Hat SUSE CentOS Ubuntu Debian Oracle AIX HP-UX Solaris
Operations Manager

X

X

X X

X

X X X X
Configuration Manager

X

X

       

X

X

X

Endpoint Protection

X X X X X X
Virtual Machine Manager

X

X

X

X