There are two Free MKV players named VLC Media Player and MPlayer OS X. These excellent and free multimedia players can tackle many of the video formats unknown to QuickTime. They contain a decoder to decode the .mkv files. They will also play many of the formats you will find inside the .mkv file.

Mac MKV Player Freeware (1) - VLC
Using VideoLAN VLC Media Player - VLC player supports all the video and audio streams you will find in an MKV file. You need not to install any codecs or filters to play .mkv files.

About VLC Media Player
VLC media player is a highly portable multimedia player for various audio and video formats (MPEG-1, MPEG-2, MPEG-4, DivX, MKV, MP3, OGG, ...) as well as DVDs, VCDs, and various streaming protocols. It can also be used as a server to stream in unicast or multicast in IPv4 or IPv6 on a high-bandwidth network. It doesn’t need any external codec or program to work.

Free Download VLC Media Player

Free MKV Player for Mac (2) - MPlayer OS X

Media Player Classic has a built in DVD player with real-time zoom, support for AVI subtitles, QuickTime and RealVideo support (requires QuickTime and/or Real Player); built-in MPEG2/SVCD/DVD codec. To play mkv file you need to install updated Matroska Pack. To do so follow simple steps below:

Open Media Player Classic and navigate to View –> Options then Select Internal Filters situated on the left and then uncheck to disable Matroska. Now Install the Matroska Pack. You should now be able to play the mkv files.

Free Download MPlayer OS X

What’s MKV format

MKV is a container format developed by Matroska. It can hold different types of video and audio. For example, there might be an .avi file contained within the .mkv file.

The Matroska Multimedia Container is an open standard free Container format, a file format that can hold an unlimited number of video, audio, picture or subtitle tracks inside a single file. It is intended to serve as a universal format for storing common multimedia content, like movies or TV shows. Matroska is similar in conception to other containers like AVI, MP4 or ASF, but is entirely open in specification, with implementations consisting mostly of open source software. Matroska file types are .MKV for video (with subtitles and audio), .MKA for audio-only files and .MKS for subtitles only. High Definition movies in 720 or 1080 pixel width format are often encoded and packaged as a Matroska format video with a .mkv file extension.

Matroska file types
.mkv : Generally video files, as well those containing audio ( movies ) or video only.
.mka : audio only files, can contain any supported audio compression format, such as MP2, MP3, Vorbis, AAC, AC3, DTS, PCM and soon MPC ( musepack ).
.mks : a so called ‘elementary’ matroska stream containing any subtitles stream.

Resource: http://aimersoft-mac.com/articles/free-play-mkv-on-mac.html